2024-02-18T11:37:36.253 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-18T11:37:36.253 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --verbose --name vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} --archive /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564390 -- /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564390/orig.config.yaml 2024-02-18T11:37:36.297 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T11:37:36.391 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564390 branch: wip-vshankar-testing-20240217.015652 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: vshankar@redhat.com first_in_suite: false job_id: '7564390' kernel: client: sha1: distro flavor: default kdb: true sha1: 8d59ab0bd93c806231018a6729893bf309624262 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240217.015652 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1c920afa8316ea21877f3186850254947531ea22 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1c920afa8316ea21877f3186850254947531ea22 kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240217.015652 sha1: 1c920afa8316ea21877f3186850254947531ea22 owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 218 sha1: 1c920afa8316ea21877f3186850254947531ea22 sleep_before_teardown: 0 subset: 793/1024 suite: fs suite_branch: wip-vshankar-testing-20240217.015652 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1c920afa8316ea21877f3186850254947531ea22 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDF9hxeylZvSIybdPlGqIGIbFHyNqvDdbsuE5RWtwdB3vgNLRp5/nPhyR3wlP1EHgKAX/1YAkAH6Q8yNAcW5Wo4= smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCibNCnHfCeF3XWLyecPEfPfJjG6P4jVBVwWVy1KdM1d0tEZbwJj+cQYNBOts2ON6zdY0t2gzEntEL3CWvXKVo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-17_05:57:56 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-18T11:37:36.391 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa; will attempt to use it 2024-02-18T11:37:36.392 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa/tasks 2024-02-18T11:37:36.392 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-18T11:37:36.393 INFO:teuthology.task.internal:Checking packages... 2024-02-18T11:37:36.417 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1c920afa8316ea21877f3186850254947531ea22' 2024-02-18T11:37:36.417 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-18T11:37:36.417 INFO:teuthology.packaging:ref: None 2024-02-18T11:37:36.417 INFO:teuthology.packaging:tag: None 2024-02-18T11:37:36.417 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240217.015652 2024-02-18T11:37:36.417 INFO:teuthology.packaging:sha1: 1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:37:36.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240217.015652 2024-02-18T11:37:36.560 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1555.g1c920afa 2024-02-18T11:37:36.561 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-18T11:37:36.568 INFO:teuthology.task.internal:no buildpackages task found 2024-02-18T11:37:36.568 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-18T11:37:36.631 INFO:teuthology.task.internal:Saving configuration 2024-02-18T11:37:36.648 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-18T11:37:36.654 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T11:37:36.678 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:32:33.422755', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDF9hxeylZvSIybdPlGqIGIbFHyNqvDdbsuE5RWtwdB3vgNLRp5/nPhyR3wlP1EHgKAX/1YAkAH6Q8yNAcW5Wo4='} 2024-02-18T11:37:36.697 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564390', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 11:32:33.421038', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCibNCnHfCeF3XWLyecPEfPfJjG6P4jVBVwWVy1KdM1d0tEZbwJj+cQYNBOts2ON6zdY0t2gzEntEL3CWvXKVo='} 2024-02-18T11:37:36.698 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-18T11:37:36.703 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-02-18T11:37:36.704 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-02-18T11:37:36.704 INFO:teuthology.run_tasks:Running task console_log... 2024-02-18T11:37:36.780 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7bea34b160>, signals=[15]) 2024-02-18T11:37:36.781 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-18T11:37:36.787 INFO:teuthology.task.internal:Opening connections... 2024-02-18T11:37:36.787 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:37:36.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:37:36.881 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:37:36.882 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:37:36.972 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-18T11:37:36.979 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-02-18T11:37:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-02-18T11:37:36.997 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="9" 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="9" 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el9" 2024-02-18T11:37:37.054 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:LOGO="fedora-logo-icon" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T11:37:37.055 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T11:37:37.056 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-02-18T11:37:37.078 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-02-18T11:37:37.096 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-02-18T11:37:37.096 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-02-18T11:37:37.153 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-18T11:37:37.154 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-02-18T11:37:37.154 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-18T11:37:37.154 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-18T11:37:37.154 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-18T11:37:37.154 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-02-18T11:37:37.178 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-18T11:37:37.187 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-18T11:37:37.193 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-18T11:37:37.193 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-02-18T11:37:37.195 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-02-18T11:37:37.212 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-18T11:37:37.218 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-18T11:37:37.218 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-02-18T11:37:37.253 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-02-18T11:37:37.344 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-18T11:37:37.401 INFO:teuthology.run_tasks:Running task kernel... 2024-02-18T11:37:37.415 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8d59ab0bd93c806231018a6729893bf309624262'} 2024-02-18T11:37:37.415 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-02-18T11:37:37.416 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-02-18T11:37:37.416 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T11:37:37.416 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T11:37:37.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:37:37.432 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-02-18T11:37:37.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:37:37.434 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-02-18T11:37:37.490 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-361.el9.x86_64 2024-02-18T11:37:37.490 INFO:teuthology.task.kernel:Running kernel on smithi116: 5.14.0-361.el9.x86_64 2024-02-18T11:37:37.490 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum install -y kernel 2024-02-18T11:37:37.491 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-361.el9.x86_64 2024-02-18T11:37:37.492 INFO:teuthology.task.kernel:Running kernel on smithi143: 5.14.0-361.el9.x86_64 2024-02-18T11:37:37.492 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum install -y kernel 2024-02-18T11:37:39.539 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-18T11:37:39.619 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-18T11:37:41.236 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-02-18T11:37:41.248 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-02-18T11:37:42.776 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-02-18T11:37:43.130 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-02-18T11:37:44.819 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-02-18T11:37:47.064 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-02-18T11:37:50.261 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.1 MB 00:01 2024-02-18T11:37:51.911 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2024-02-18T11:37:51.998 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.1 MB 00:01 2024-02-18T11:37:53.574 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 15 kB 00:00 2024-02-18T11:37:53.655 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-18T11:37:55.336 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-18T11:37:58.529 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-02-18T11:38:00.117 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-02-18T11:38:00.778 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-18T11:38:00.778 INFO:teuthology.orchestra.run.smithi116.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-18T11:38:00.864 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-02-18T11:38:00.875 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-02-18T11:38:00.875 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-02-18T11:38:00.875 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout:Install 4 Packages 2024-02-18T11:38:00.876 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:00.877 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 100 M 2024-02-18T11:38:00.877 INFO:teuthology.orchestra.run.smithi116.stdout:Installed size: 123 M 2024-02-18T11:38:00.877 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-02-18T11:38:01.136 INFO:teuthology.orchestra.run.smithi116.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 23 MB/s | 6.1 MB 00:00 2024-02-18T11:38:01.481 INFO:teuthology.orchestra.run.smithi116.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 35 MB/s | 21 MB 00:00 2024-02-18T11:38:01.848 INFO:teuthology.orchestra.run.smithi116.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 40 MB/s | 39 MB 00:00 2024-02-18T11:38:02.148 INFO:teuthology.orchestra.run.smithi116.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 33 MB/s | 33 MB 00:01 2024-02-18T11:38:02.151 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:38:02.151 INFO:teuthology.orchestra.run.smithi116.stdout:Total 78 MB/s | 100 MB 00:01 2024-02-18T11:38:02.151 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-02-18T11:38:02.324 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-18T11:38:02.324 INFO:teuthology.orchestra.run.smithi143.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-18T11:38:02.416 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-02-18T11:38:02.425 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout:================================================================================ 2024-02-18T11:38:02.426 INFO:teuthology.orchestra.run.smithi143.stdout:Install 4 Packages 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 100 M 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi143.stdout:Installed size: 123 M 2024-02-18T11:38:02.427 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-02-18T11:38:02.518 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-02-18T11:38:02.518 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-02-18T11:38:02.718 INFO:teuthology.orchestra.run.smithi143.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 21 MB/s | 6.1 MB 00:00 2024-02-18T11:38:03.060 INFO:teuthology.orchestra.run.smithi143.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 33 MB/s | 21 MB 00:00 2024-02-18T11:38:03.486 INFO:teuthology.orchestra.run.smithi143.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 44 MB/s | 33 MB 00:00 2024-02-18T11:38:03.551 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-02-18T11:38:03.551 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-02-18T11:38:03.961 INFO:teuthology.orchestra.run.smithi143.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 26 MB/s | 39 MB 00:01 2024-02-18T11:38:03.965 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:38:03.966 INFO:teuthology.orchestra.run.smithi143.stdout:Total 65 MB/s | 100 MB 00:01 2024-02-18T11:38:03.966 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-02-18T11:38:04.356 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-02-18T11:38:04.356 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-02-18T11:38:05.392 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-02-18T11:38:05.392 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-02-18T11:38:05.956 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-02-18T11:38:06.647 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-18T11:38:06.773 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:38:07.794 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-02-18T11:38:08.468 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-18T11:38:08.605 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:38:09.532 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:38:09.709 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:38:11.385 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:38:11.586 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:38:14.431 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:38:14.797 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:16.274 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:38:16.390 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:19.537 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:21.125 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:48.764 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:48.764 INFO:teuthology.orchestra.run.smithi116.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-18T11:38:48.764 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:48.835 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:49.258 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:49.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-18T11:38:49.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:38:49.258 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:38:49.694 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:49.695 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-02-18T11:38:50.166 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T11:38:50.307 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:50.308 INFO:teuthology.orchestra.run.smithi143.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-18T11:38:50.308 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:50.344 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-18T11:38:50.345 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-18T11:38:50.345 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-18T11:38:50.346 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T11:38:50.346 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T11:38:50.346 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-18T11:38:50.346 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi116.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T11:38:50.346 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T11:38:50.379 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:50.453 INFO:teuthology.orchestra.run.smithi116.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-18T11:38:50.453 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-18T11:38:50.453 DEBUG:teuthology.orchestra.run.smithi116:> sudo rpm -qi grub2-tools 2024-02-18T11:38:50.487 INFO:teuthology.orchestra.run.smithi116.stdout:Name : grub2-tools 2024-02-18T11:38:50.487 INFO:teuthology.orchestra.run.smithi116.stdout:Epoch : 1 2024-02-18T11:38:50.487 INFO:teuthology.orchestra.run.smithi116.stdout:Version : 2.06 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Release : 68.el9 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Architecture: x86_64 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Group : Unspecified 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Size : 8264254 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:License : GPLv3+ 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Packager : builder@centos.org 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Vendor : CentOS 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-18T11:38:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:Summary : Support tools for GRUB. 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout:Description : 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout:hardware devices. 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:38:50.489 INFO:teuthology.orchestra.run.smithi116.stdout:This subpackage provides tools for support of all platforms. 2024-02-18T11:38:50.490 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-18T11:38:50.490 INFO:teuthology.task.kernel:Updating grub on smithi116 to boot 5.14.0-421.el9.x86_64 2024-02-18T11:38:50.490 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-18T11:38:50.764 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:50.764 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-18T11:38:50.764 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-18T11:38:50.764 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-18T11:38:51.117 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-18T11:38:51.223 INFO:teuthology.orchestra.run.smithi143.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-18T11:38:51.224 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:51.224 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-02-18T11:38:51.750 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T11:38:51.878 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-18T11:38:51.879 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-18T11:38:51.879 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-18T11:38:51.879 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T11:38:51.879 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T11:38:51.879 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-02-18T11:38:51.879 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T11:38:51.879 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q kernel | sort -rV | head -n 1 2024-02-18T11:38:51.987 INFO:teuthology.orchestra.run.smithi143.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-18T11:38:51.988 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-18T11:38:51.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo rpm -qi grub2-tools 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:Name : grub2-tools 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:Epoch : 1 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:Version : 2.06 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:Release : 68.el9 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: x86_64 2024-02-18T11:38:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Group : Unspecified 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Size : 8264254 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:License : GPLv3+ 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Packager : builder@centos.org 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Vendor : CentOS 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Summary : Support tools for GRUB. 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:Description : 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:52.023 INFO:teuthology.orchestra.run.smithi143.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi143.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi143.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi143.stdout:hardware devices. 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:38:52.024 INFO:teuthology.orchestra.run.smithi143.stdout:This subpackage provides tools for support of all platforms. 2024-02-18T11:38:52.025 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-18T11:38:52.025 INFO:teuthology.task.kernel:Updating grub on smithi143 to boot 5.14.0-421.el9.x86_64 2024-02-18T11:38:52.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-18T11:38:52.564 INFO:teuthology.orchestra.run.smithi116.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-18T11:38:52.600 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-02-18T11:38:52.602 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-02-18T11:38:52.617 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.4n4FH3rBTQ 2024-02-18T11:38:52.617 DEBUG:teuthology.orchestra.run.smithi116:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4n4FH3rBTQ 2024-02-18T11:38:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-02-18T11:38:52.685 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.4n4FH3rBTQ 2024-02-18T11:38:52.821 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.4n4FH3rBTQ is 6KB 2024-02-18T11:38:52.840 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.4n4FH3rBTQ 2024-02-18T11:38:52.853 DEBUG:teuthology.orchestra.run.smithi116:> sudo /bin/ls /boot/loader/entries || true 2024-02-18T11:38:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-18T11:38:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-18T11:38:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-18T11:38:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-18T11:38:52.930 DEBUG:teuthology.orchestra.run.smithi116:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-18T11:38:53.089 DEBUG:teuthology.orchestra.run.smithi116:> sudo shutdown -r now 2024-02-18T11:38:54.008 INFO:teuthology.orchestra.run.smithi143.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-18T11:38:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-02-18T11:38:54.042 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-02-18T11:38:54.057 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.Q6LvWVOpP4 2024-02-18T11:38:54.057 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Q6LvWVOpP4 2024-02-18T11:38:54.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.Q6LvWVOpP4 2024-02-18T11:38:54.264 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.Q6LvWVOpP4 is 6KB 2024-02-18T11:38:54.313 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.Q6LvWVOpP4 2024-02-18T11:38:54.328 DEBUG:teuthology.orchestra.run.smithi143:> sudo /bin/ls /boot/loader/entries || true 2024-02-18T11:38:54.392 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-18T11:38:54.392 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-18T11:38:54.392 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-18T11:38:54.392 INFO:teuthology.orchestra.run.smithi143.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-18T11:38:54.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-18T11:38:54.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2024-02-18T11:39:23.120 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-18T11:39:23.121 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:39:23.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:39:24.550 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-02-18T11:39:24.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-02-18T11:39:24.550 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:39:41.500 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-02-18T11:39:43.036 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-02-18T11:39:50.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:39:50.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:39:52.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-02-18T11:39:52.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:39:53.564 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-02-18T11:39:55.100 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-02-18T11:40:05.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:40:05.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:40:07.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-02-18T11:40:07.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:40:23.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-02-18T11:40:38.999 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:40:39.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:40:39.044 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-02-18T11:40:39.297 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-02-18T11:40:39.575 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-02-18T11:40:39.575 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-421.el9.x86_64"... 2024-02-18T11:40:39.575 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-02-18T11:40:39.598 INFO:teuthology.orchestra.run.smithi143.stdout:5.14.0-421.el9.x86_64 2024-02-18T11:40:39.598 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-18T11:40:39.598 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T11:40:39.599 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-18T11:40:39.763 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:40:39.763 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-421.el9.x86_64"... 2024-02-18T11:40:39.764 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-02-18T11:40:39.777 INFO:teuthology.orchestra.run.smithi116.stdout:5.14.0-421.el9.x86_64 2024-02-18T11:40:39.778 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-18T11:40:39.778 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T11:40:39.778 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-18T11:40:40.599 DEBUG:teuthology.parallel:result is None 2024-02-18T11:40:40.778 DEBUG:teuthology.parallel:result is None 2024-02-18T11:40:40.779 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-18T11:40:40.788 INFO:teuthology.task.internal:Creating test directory... 2024-02-18T11:40:40.788 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T11:40:40.792 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T11:40:40.812 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-18T11:40:40.819 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-18T11:40:40.828 INFO:teuthology.task.internal:Creating archive directory... 2024-02-18T11:40:40.828 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T11:40:40.855 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T11:40:40.893 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-18T11:40:40.901 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-18T11:40:40.901 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T11:40:40.930 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T11:40:41.002 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:40:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:40:41.064 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:40:41.087 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T11:40:41.088 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-18T11:40:41.097 INFO:teuthology.task.internal:Configuring sudo... 2024-02-18T11:40:41.097 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T11:40:41.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T11:40:41.142 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-18T11:40:41.164 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-18T11:40:41.164 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T11:40:41.185 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T11:40:41.206 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:40:41.302 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:40:41.353 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:40:41.436 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:40:41.478 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:40:41.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T11:40:41.545 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:40:41.587 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T11:40:41.635 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:40:41.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T11:40:41.759 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:40:41.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T11:40:41.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-02-18T11:40:41.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-02-18T11:40:41.905 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T11:40:41.945 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-18T11:40:42.349 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-18T11:40:42.358 INFO:teuthology.task.internal:Starting timer... 2024-02-18T11:40:42.359 INFO:teuthology.run_tasks:Running task pcp... 2024-02-18T11:40:42.377 INFO:teuthology.run_tasks:Running task selinux... 2024-02-18T11:40:42.393 DEBUG:teuthology.orchestra.run.smithi116:> sudo service auditd rotate 2024-02-18T11:40:42.444 INFO:teuthology.orchestra.run.smithi116.stdout:Rotating logs: 2024-02-18T11:40:42.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-02-18T11:40:42.498 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-02-18T11:40:42.499 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-18T11:40:42.500 DEBUG:teuthology.orchestra.run.smithi116:> /usr/sbin/getenforce 2024-02-18T11:40:42.520 INFO:teuthology.orchestra.run.smithi116.stdout:Permissive 2024-02-18T11:40:42.521 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-02-18T11:40:42.542 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-02-18T11:40:42.543 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi116.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi143.front.sepia.ceph.com': 'permissive'} 2024-02-18T11:40:42.543 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T11:40:42.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:40:42.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-18T11:40:42.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:40:42.617 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-18T11:40:42.617 DEBUG:teuthology.orchestra.run.smithi116:> sudo /usr/sbin/setenforce permissive 2024-02-18T11:40:42.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-02-18T11:40:42.702 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-18T11:40:42.714 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-18T11:40:42.732 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-18T11:40:42.733 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi116.front.sepia.ceph.com,smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-18T11:44:53.751 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi116.front.sepia.ceph.com'), Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2024-02-18T11:44:53.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:44:53.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:44:53.844 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-02-18T11:44:53.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-02-18T11:44:53.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-02-18T11:44:53.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T11:44:54.013 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-02-18T11:44:54.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-02-18T11:44:54.093 INFO:teuthology.run_tasks:Running task clock... 2024-02-18T11:44:54.105 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-18T11:44:54.105 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T11:44:54.105 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T11:44:54.108 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T11:44:54.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T11:44:54.136 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T11:44:54.152 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T11:44:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-18T11:44:54.181 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: ntpd: command not found 2024-02-18T11:44:54.183 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-18T11:44:54.194 INFO:teuthology.orchestra.run.smithi116.stdout:506 Cannot talk to daemon 2024-02-18T11:44:54.211 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T11:44:54.212 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-02-18T11:44:54.226 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-02-18T11:44:54.227 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T11:44:54.242 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-18T11:44:54.259 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-18T11:44:54.283 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-02-18T11:44:54.285 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T11:44:54.285 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-02-18T11:44:54.286 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.286 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.286 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.286 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.312 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-02-18T11:44:54.314 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-18T11:44:54.315 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-02-18T11:44:54.315 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.315 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.315 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.315 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-18T11:44:54.316 INFO:teuthology.run_tasks:Running task install... 2024-02-18T11:44:54.324 DEBUG:teuthology.task.install:project ceph 2024-02-18T11:44:54.324 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'}} 2024-02-18T11:44:54.324 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'} 2024-02-18T11:44:54.324 INFO:teuthology.task.install:Using flavor: default 2024-02-18T11:44:54.330 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-18T11:44:54.330 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-18T11:44:54.330 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'tag': None, 'wait_for_package': False} 2024-02-18T11:44:54.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:44:54.333 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'tag': None, 'wait_for_package': False} 2024-02-18T11:44:54.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:44:54.492 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/ 2024-02-18T11:44:54.492 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1555.g1c920afa 2024-02-18T11:44:54.550 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/ 2024-02-18T11:44:54.550 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1555.g1c920afa 2024-02-18T11:44:54.668 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T11:44:54.668 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:44:54.668 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T11:44:54.696 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-18T11:44:54.696 DEBUG:teuthology.orchestra.run.smithi116:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c920afa8316ea21877f3186850254947531ea22/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T11:44:54.736 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240217.015652/1c920afa8316ea21877f3186850254947531ea22/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-18T11:44:54.736 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:44:54.737 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-18T11:44:54.764 DEBUG:teuthology.orchestra.run.smithi116:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T11:44:54.765 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-18T11:44:54.766 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c920afa8316ea21877f3186850254947531ea22/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-18T11:44:54.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-18T11:44:54.844 DEBUG:teuthology.orchestra.run.smithi116:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T11:44:54.911 INFO:teuthology.orchestra.run.smithi116.stdout:check_obsoletes = 1 2024-02-18T11:44:54.912 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum clean all 2024-02-18T11:44:54.915 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-18T11:44:54.982 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-02-18T11:44:54.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-02-18T11:44:55.161 INFO:teuthology.orchestra.run.smithi116.stdout:74 files removed 2024-02-18T11:44:55.188 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-18T11:44:55.234 INFO:teuthology.orchestra.run.smithi143.stdout:74 files removed 2024-02-18T11:44:55.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-18T11:44:55.811 INFO:teuthology.orchestra.run.smithi116.stdout:ceph packages for x86_64 265 kB/s | 82 kB 00:00 2024-02-18T11:44:56.187 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 134 kB/s | 82 kB 00:00 2024-02-18T11:44:56.262 INFO:teuthology.orchestra.run.smithi116.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-02-18T11:44:56.587 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 28 kB/s | 10 kB 00:00 2024-02-18T11:44:56.721 INFO:teuthology.orchestra.run.smithi116.stdout:ceph source packages 4.5 kB/s | 1.9 kB 00:00 2024-02-18T11:44:56.896 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 7.0 kB/s | 1.9 kB 00:00 2024-02-18T11:44:57.089 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - AppStream 20 MB/s | 6.7 MB 00:00 2024-02-18T11:44:57.257 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - AppStream 21 MB/s | 6.7 MB 00:00 2024-02-18T11:44:58.524 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-02-18T11:44:58.632 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS-9 - BaseOS 7.9 MB/s | 2.0 MB 00:00 2024-02-18T11:44:59.698 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-02-18T11:45:00.389 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-02-18T11:45:01.541 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-02-18T11:45:02.265 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-02-18T11:45:05.741 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.1 MB 00:00 2024-02-18T11:45:06.257 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-02-18T11:45:07.125 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 15 kB 00:00 2024-02-18T11:45:08.226 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-02-18T11:45:08.904 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-18T11:45:10.752 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 20 MB 00:02 2024-02-18T11:45:13.718 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-02-18T11:45:15.693 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-18T11:45:16.270 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:45:16.271 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:45:16.284 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-18T11:45:16.285 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-18T11:45:16.286 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-18T11:45:16.287 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-18T11:45:16.289 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-18T11:45:16.292 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-18T11:45:16.293 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-18T11:45:16.374 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-02-18T11:45:16.391 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-02-18T11:45:16.391 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-02-18T11:45:16.391 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-02-18T11:45:16.391 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-02-18T11:45:16.391 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-18T11:45:16.391 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 6.6 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.3 M 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 886 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 145 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 1.5 M 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 138 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 1.8 M 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 7.4 M 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 52 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 9.8 M 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 47 M 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 264 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 748 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 220 k 2024-02-18T11:45:16.392 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 27 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 18 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 31 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 742 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 127 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-18T11:45:16.393 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 161 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 320 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 298 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 99 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 85 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 171 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-18T11:45:16.394 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.4 M 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:45:16.395 INFO:teuthology.orchestra.run.smithi143.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-18T11:45:16.396 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 20 M 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 31 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 2.1 M 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 248 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 4.7 M 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 17 M 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 15 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 25 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-18T11:45:16.397 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 163 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 499 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.1 M 2024-02-18T11:45:16.398 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-18T11:45:16.399 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 45 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 131 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-18T11:45:16.400 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-18T11:45:16.401 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-18T11:45:16.402 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-18T11:45:16.403 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-18T11:45:16.404 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:45:16.405 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-02-18T11:45:16.405 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================================ 2024-02-18T11:45:16.405 INFO:teuthology.orchestra.run.smithi143.stdout:Install 150 Packages 2024-02-18T11:45:16.405 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 26 Packages 2024-02-18T11:45:16.405 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:45:16.405 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 223 M 2024-02-18T11:45:16.406 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-02-18T11:45:16.952 INFO:teuthology.orchestra.run.smithi143.stdout:(1/176): ceph-19.0.0-1555.g1c920afa.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-02-18T11:45:17.211 INFO:teuthology.orchestra.run.smithi143.stdout:(2/176): ceph-fuse-19.0.0-1555.g1c920afa.el9.x8 3.4 MB/s | 886 kB 00:00 2024-02-18T11:45:17.278 INFO:teuthology.orchestra.run.smithi143.stdout:(3/176): ceph-immutable-object-cache-19.0.0-155 2.1 MB/s | 145 kB 00:00 2024-02-18T11:45:17.603 INFO:teuthology.orchestra.run.smithi143.stdout:(4/176): ceph-mds-19.0.0-1555.g1c920afa.el9.x86 6.5 MB/s | 2.1 MB 00:00 2024-02-18T11:45:17.728 INFO:teuthology.orchestra.run.smithi143.stdout:(5/176): ceph-base-19.0.0-1555.g1c920afa.el9.x8 5.6 MB/s | 5.3 MB 00:00 2024-02-18T11:45:17.870 INFO:teuthology.orchestra.run.smithi143.stdout:(6/176): ceph-mgr-19.0.0-1555.g1c920afa.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-02-18T11:45:18.304 INFO:teuthology.orchestra.run.smithi143.stdout:(7/176): ceph-mon-19.0.0-1555.g1c920afa.el9.x86 8.1 MB/s | 4.7 MB 00:00 2024-02-18T11:45:18.351 INFO:teuthology.orchestra.run.smithi116.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:45:18.352 INFO:teuthology.orchestra.run.smithi116.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-18T11:45:18.366 INFO:teuthology.orchestra.run.smithi116.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-18T11:45:18.367 INFO:teuthology.orchestra.run.smithi116.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-18T11:45:18.368 INFO:teuthology.orchestra.run.smithi116.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-18T11:45:18.369 INFO:teuthology.orchestra.run.smithi116.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-18T11:45:18.371 INFO:teuthology.orchestra.run.smithi116.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-18T11:45:18.376 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-18T11:45:18.376 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-18T11:45:18.454 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-02-18T11:45:18.470 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================================ 2024-02-18T11:45:18.470 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-02-18T11:45:18.470 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================================ 2024-02-18T11:45:18.470 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 6.6 k 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.3 M 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 886 k 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 145 k 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 1.5 M 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 138 k 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 1.8 M 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 7.4 M 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 52 k 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 9.8 M 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 47 M 2024-02-18T11:45:18.471 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 264 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 748 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 220 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 27 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 18 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 31 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 742 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 127 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-18T11:45:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 161 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 320 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 298 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 99 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 85 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 171 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrading: 2024-02-18T11:45:18.473 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-18T11:45:18.474 INFO:teuthology.orchestra.run.smithi116.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: librados2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.4 M 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 3.1 M 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-18T11:45:18.475 INFO:teuthology.orchestra.run.smithi116.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 20 M 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 31 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 2.1 M 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 248 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 4.7 M 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 17 M 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1555.g1c920afa.el9 ceph-noarch 15 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 25 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-18T11:45:18.476 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 163 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 499 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-18T11:45:18.477 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2 x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 5.1 M 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-18T11:45:18.478 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 45 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common x86_64 2:19.0.0-1555.g1c920afa.el9 ceph 131 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-18T11:45:18.479 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-18T11:45:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-18T11:45:18.481 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-18T11:45:18.482 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-18T11:45:18.483 INFO:teuthology.orchestra.run.smithi116.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:Installing weak dependencies: 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================================ 2024-02-18T11:45:18.484 INFO:teuthology.orchestra.run.smithi116.stdout:Install 150 Packages 2024-02-18T11:45:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrade 26 Packages 2024-02-18T11:45:18.485 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:45:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 223 M 2024-02-18T11:45:18.485 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-02-18T11:45:19.146 INFO:teuthology.orchestra.run.smithi143.stdout:(8/176): ceph-common-19.0.0-1555.g1c920afa.el9. 8.4 MB/s | 20 MB 00:02 2024-02-18T11:45:19.204 INFO:teuthology.orchestra.run.smithi143.stdout:(9/176): ceph-selinux-19.0.0-1555.g1c920afa.el9 434 kB/s | 25 kB 00:00 2024-02-18T11:45:19.336 INFO:teuthology.orchestra.run.smithi116.stdout:(1/176): ceph-19.0.0-1555.g1c920afa.el9.x86_64. 32 kB/s | 6.6 kB 00:00 2024-02-18T11:45:19.346 INFO:teuthology.orchestra.run.smithi143.stdout:(10/176): ceph-radosgw-19.0.0-1555.g1c920afa.el 9.4 MB/s | 9.8 MB 00:01 2024-02-18T11:45:19.421 INFO:teuthology.orchestra.run.smithi143.stdout:(11/176): cephfs-mirror-19.0.0-1555.g1c920afa.e 2.9 MB/s | 220 kB 00:00 2024-02-18T11:45:19.480 INFO:teuthology.orchestra.run.smithi143.stdout:(12/176): cephfs-shell-19.0.0-1555.g1c920afa.el 465 kB/s | 27 kB 00:00 2024-02-18T11:45:19.538 INFO:teuthology.orchestra.run.smithi143.stdout:(13/176): libcephfs-devel-19.0.0-1555.g1c920afa 536 kB/s | 31 kB 00:00 2024-02-18T11:45:19.680 INFO:teuthology.orchestra.run.smithi143.stdout:(14/176): libcephfs2-19.0.0-1555.g1c920afa.el9. 5.1 MB/s | 742 kB 00:00 2024-02-18T11:45:19.755 INFO:teuthology.orchestra.run.smithi143.stdout:(15/176): libcephsqlite-19.0.0-1555.g1c920afa.e 2.1 MB/s | 163 kB 00:00 2024-02-18T11:45:19.886 INFO:teuthology.orchestra.run.smithi116.stdout:(2/176): ceph-fuse-19.0.0-1555.g1c920afa.el9.x8 1.6 MB/s | 886 kB 00:00 2024-02-18T11:45:19.961 INFO:teuthology.orchestra.run.smithi116.stdout:(3/176): ceph-immutable-object-cache-19.0.0-155 1.9 MB/s | 145 kB 00:00 2024-02-18T11:45:19.972 INFO:teuthology.orchestra.run.smithi143.stdout:(16/176): ceph-osd-19.0.0-1555.g1c920afa.el9.x8 8.0 MB/s | 17 MB 00:02 2024-02-18T11:45:19.997 INFO:teuthology.orchestra.run.smithi143.stdout:(17/176): librados-devel-19.0.0-1555.g1c920afa. 525 kB/s | 127 kB 00:00 2024-02-18T11:45:20.048 INFO:teuthology.orchestra.run.smithi143.stdout:(18/176): libradosstriper1-19.0.0-1555.g1c920af 6.5 MB/s | 499 kB 00:00 2024-02-18T11:45:20.131 INFO:teuthology.orchestra.run.smithi143.stdout:(19/176): python3-ceph-argparse-19.0.0-1555.g1c 541 kB/s | 45 kB 00:00 2024-02-18T11:45:20.223 INFO:teuthology.orchestra.run.smithi143.stdout:(20/176): python3-ceph-common-19.0.0-1555.g1c92 1.4 MB/s | 131 kB 00:00 2024-02-18T11:45:20.315 INFO:teuthology.orchestra.run.smithi143.stdout:(21/176): python3-cephfs-19.0.0-1555.g1c920afa. 1.7 MB/s | 161 kB 00:00 2024-02-18T11:45:20.416 INFO:teuthology.orchestra.run.smithi143.stdout:(22/176): python3-rados-19.0.0-1555.g1c920afa.e 3.1 MB/s | 320 kB 00:00 2024-02-18T11:45:20.516 INFO:teuthology.orchestra.run.smithi143.stdout:(23/176): python3-rbd-19.0.0-1555.g1c920afa.el9 2.9 MB/s | 298 kB 00:00 2024-02-18T11:45:20.608 INFO:teuthology.orchestra.run.smithi143.stdout:(24/176): python3-rgw-19.0.0-1555.g1c920afa.el9 1.1 MB/s | 99 kB 00:00 2024-02-18T11:45:20.683 INFO:teuthology.orchestra.run.smithi143.stdout:(25/176): librgw2-19.0.0-1555.g1c920afa.el9.x86 7.5 MB/s | 5.1 MB 00:00 2024-02-18T11:45:20.700 INFO:teuthology.orchestra.run.smithi143.stdout:(26/176): rbd-fuse-19.0.0-1555.g1c920afa.el9.x8 928 kB/s | 85 kB 00:00 2024-02-18T11:45:20.720 INFO:teuthology.orchestra.run.smithi116.stdout:(4/176): ceph-base-19.0.0-1555.g1c920afa.el9.x8 3.4 MB/s | 5.3 MB 00:01 2024-02-18T11:45:20.783 INFO:teuthology.orchestra.run.smithi143.stdout:(27/176): rbd-nbd-19.0.0-1555.g1c920afa.el9.x86 2.0 MB/s | 171 kB 00:00 2024-02-18T11:45:20.870 INFO:teuthology.orchestra.run.smithi116.stdout:(5/176): ceph-mds-19.0.0-1555.g1c920afa.el9.x86 2.3 MB/s | 2.1 MB 00:00 2024-02-18T11:45:20.884 INFO:teuthology.orchestra.run.smithi143.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1555.g 314 kB/s | 31 kB 00:00 2024-02-18T11:45:21.103 INFO:teuthology.orchestra.run.smithi116.stdout:(6/176): ceph-mgr-19.0.0-1555.g1c920afa.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-02-18T11:45:21.126 INFO:teuthology.orchestra.run.smithi143.stdout:(29/176): rbd-mirror-19.0.0-1555.g1c920afa.el9. 7.0 MB/s | 3.1 MB 00:00 2024-02-18T11:45:21.276 INFO:teuthology.orchestra.run.smithi143.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1555.g1c920af 352 kB/s | 138 kB 00:00 2024-02-18T11:45:21.701 INFO:teuthology.orchestra.run.smithi143.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1555.g1c920 3.1 MB/s | 1.8 MB 00:00 2024-02-18T11:45:21.745 INFO:teuthology.orchestra.run.smithi116.stdout:(7/176): ceph-mon-19.0.0-1555.g1c920afa.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-02-18T11:45:21.852 INFO:teuthology.orchestra.run.smithi143.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1555.g1c 1.6 MB/s | 248 kB 00:00 2024-02-18T11:45:21.943 INFO:teuthology.orchestra.run.smithi143.stdout:(33/176): ceph-mgr-rook-19.0.0-1555.g1c920afa.e 572 kB/s | 52 kB 00:00 2024-02-18T11:45:22.035 INFO:teuthology.orchestra.run.smithi143.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1555.g1 167 kB/s | 15 kB 00:00 2024-02-18T11:45:22.210 INFO:teuthology.orchestra.run.smithi143.stdout:(35/176): ceph-volume-19.0.0-1555.g1c920afa.el9 1.5 MB/s | 264 kB 00:00 2024-02-18T11:45:22.553 INFO:teuthology.orchestra.run.smithi143.stdout:(36/176): cephadm-19.0.0-1555.g1c920afa.el9.noa 2.1 MB/s | 748 kB 00:00 2024-02-18T11:45:22.694 INFO:teuthology.orchestra.run.smithi143.stdout:(37/176): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-02-18T11:45:22.719 INFO:teuthology.orchestra.run.smithi143.stdout:(38/176): cephfs-top-19.0.0-1555.g1c920afa.el9. 109 kB/s | 18 kB 00:00 2024-02-18T11:45:22.861 INFO:teuthology.orchestra.run.smithi143.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 6.4 MB/s | 920 kB 00:00 2024-02-18T11:45:22.895 INFO:teuthology.orchestra.run.smithi143.stdout:(40/176): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-02-18T11:45:22.920 INFO:teuthology.orchestra.run.smithi143.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-02-18T11:45:22.946 INFO:teuthology.orchestra.run.smithi143.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 406 kB/s | 20 kB 00:00 2024-02-18T11:45:22.970 INFO:teuthology.orchestra.run.smithi116.stdout:(8/176): ceph-radosgw-19.0.0-1555.g1c920afa.el9 8.0 MB/s | 9.8 MB 00:01 2024-02-18T11:45:22.971 INFO:teuthology.orchestra.run.smithi143.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-02-18T11:45:22.996 INFO:teuthology.orchestra.run.smithi143.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-02-18T11:45:23.037 INFO:teuthology.orchestra.run.smithi116.stdout:(9/176): ceph-selinux-19.0.0-1555.g1c920afa.el9 378 kB/s | 25 kB 00:00 2024-02-18T11:45:23.063 INFO:teuthology.orchestra.run.smithi143.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-02-18T11:45:23.088 INFO:teuthology.orchestra.run.smithi143.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 190 kB/s | 17 kB 00:00 2024-02-18T11:45:23.113 INFO:teuthology.orchestra.run.smithi143.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 405 kB/s | 10 kB 00:00 2024-02-18T11:45:23.138 INFO:teuthology.orchestra.run.smithi143.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-02-18T11:45:23.155 INFO:teuthology.orchestra.run.smithi143.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-18T11:45:23.189 INFO:teuthology.orchestra.run.smithi143.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-18T11:45:23.214 INFO:teuthology.orchestra.run.smithi143.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-02-18T11:45:23.231 INFO:teuthology.orchestra.run.smithi143.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 595 kB/s | 24 kB 00:00 2024-02-18T11:45:23.256 INFO:teuthology.orchestra.run.smithi143.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-02-18T11:45:23.299 INFO:teuthology.orchestra.run.smithi143.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2024-02-18T11:45:23.324 INFO:teuthology.orchestra.run.smithi143.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2024-02-18T11:45:23.349 INFO:teuthology.orchestra.run.smithi143.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 683 kB/s | 34 kB 00:00 2024-02-18T11:45:23.433 INFO:teuthology.orchestra.run.smithi143.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-02-18T11:45:23.492 INFO:teuthology.orchestra.run.smithi143.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-02-18T11:45:23.533 INFO:teuthology.orchestra.run.smithi143.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 16 MB/s | 1.5 MB 00:00 2024-02-18T11:45:23.558 INFO:teuthology.orchestra.run.smithi143.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-02-18T11:45:23.584 INFO:teuthology.orchestra.run.smithi143.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-02-18T11:45:23.629 INFO:teuthology.orchestra.run.smithi116.stdout:(10/176): ceph-common-19.0.0-1555.g1c920afa.el9 4.4 MB/s | 20 MB 00:04 2024-02-18T11:45:23.668 INFO:teuthology.orchestra.run.smithi143.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 52 MB/s | 5.7 MB 00:00 2024-02-18T11:45:23.700 INFO:teuthology.orchestra.run.smithi116.stdout:(11/176): cephfs-mirror-19.0.0-1555.g1c920afa.e 3.2 MB/s | 220 kB 00:00 2024-02-18T11:45:23.701 INFO:teuthology.orchestra.run.smithi143.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 8.8 MB/s | 1.0 MB 00:00 2024-02-18T11:45:23.726 INFO:teuthology.orchestra.run.smithi143.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-18T11:45:23.763 INFO:teuthology.orchestra.run.smithi116.stdout:(12/176): cephfs-shell-19.0.0-1555.g1c920afa.el 407 kB/s | 27 kB 00:00 2024-02-18T11:45:23.819 INFO:teuthology.orchestra.run.smithi143.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 49 MB/s | 5.8 MB 00:00 2024-02-18T11:45:23.830 INFO:teuthology.orchestra.run.smithi116.stdout:(13/176): libcephfs-devel-19.0.0-1555.g1c920afa 465 kB/s | 31 kB 00:00 2024-02-18T11:45:23.844 INFO:teuthology.orchestra.run.smithi143.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 227 kB 00:00 2024-02-18T11:45:23.869 INFO:teuthology.orchestra.run.smithi143.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 492 kB/s | 24 kB 00:00 2024-02-18T11:45:23.894 INFO:teuthology.orchestra.run.smithi143.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 360 kB/s | 18 kB 00:00 2024-02-18T11:45:23.911 INFO:teuthology.orchestra.run.smithi143.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-18T11:45:23.937 INFO:teuthology.orchestra.run.smithi143.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 755 kB/s | 32 kB 00:00 2024-02-18T11:45:24.012 INFO:teuthology.orchestra.run.smithi143.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 48 MB/s | 4.8 MB 00:00 2024-02-18T11:45:24.014 INFO:teuthology.orchestra.run.smithi116.stdout:(14/176): libcephfs2-19.0.0-1555.g1c920afa.el9. 4.0 MB/s | 742 kB 00:00 2024-02-18T11:45:24.037 INFO:teuthology.orchestra.run.smithi143.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 217 kB 00:00 2024-02-18T11:45:24.062 INFO:teuthology.orchestra.run.smithi143.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-18T11:45:24.088 INFO:teuthology.orchestra.run.smithi143.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-18T11:45:24.089 INFO:teuthology.orchestra.run.smithi116.stdout:(15/176): libcephsqlite-19.0.0-1555.g1c920afa.e 2.1 MB/s | 163 kB 00:00 2024-02-18T11:45:24.113 INFO:teuthology.orchestra.run.smithi143.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-02-18T11:45:24.138 INFO:teuthology.orchestra.run.smithi143.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-18T11:45:24.163 INFO:teuthology.orchestra.run.smithi143.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-18T11:45:24.181 INFO:teuthology.orchestra.run.smithi116.stdout:(16/176): librados-devel-19.0.0-1555.g1c920afa. 1.4 MB/s | 127 kB 00:00 2024-02-18T11:45:24.188 INFO:teuthology.orchestra.run.smithi143.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-18T11:45:24.214 INFO:teuthology.orchestra.run.smithi143.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-02-18T11:45:24.239 INFO:teuthology.orchestra.run.smithi143.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 858 kB/s | 43 kB 00:00 2024-02-18T11:45:24.323 INFO:teuthology.orchestra.run.smithi116.stdout:(17/176): libradosstriper1-19.0.0-1555.g1c920af 3.4 MB/s | 499 kB 00:00 2024-02-18T11:45:24.481 INFO:teuthology.orchestra.run.smithi143.stdout:(81/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 59 MB/s | 16 MB 00:00 2024-02-18T11:45:24.506 INFO:teuthology.orchestra.run.smithi143.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 146 kB/s | 39 kB 00:00 2024-02-18T11:45:24.531 INFO:teuthology.orchestra.run.smithi143.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 822 kB/s | 41 kB 00:00 2024-02-18T11:45:24.557 INFO:teuthology.orchestra.run.smithi143.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-18T11:45:24.574 INFO:teuthology.orchestra.run.smithi143.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 809 kB/s | 34 kB 00:00 2024-02-18T11:45:24.591 INFO:teuthology.orchestra.run.smithi143.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-02-18T11:45:24.616 INFO:teuthology.orchestra.run.smithi143.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 7.1 MB/s | 300 kB 00:00 2024-02-18T11:45:24.633 INFO:teuthology.orchestra.run.smithi143.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 56 kB 00:00 2024-02-18T11:45:24.658 INFO:teuthology.orchestra.run.smithi143.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-02-18T11:45:24.675 INFO:teuthology.orchestra.run.smithi143.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-18T11:45:24.692 INFO:teuthology.orchestra.run.smithi143.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-02-18T11:45:24.717 INFO:teuthology.orchestra.run.smithi143.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-02-18T11:45:24.743 INFO:teuthology.orchestra.run.smithi143.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-18T11:45:24.768 INFO:teuthology.orchestra.run.smithi143.stdout:(94/176): libgfortran-11.4.1-3.el9.x86_64.rpm 16 MB/s | 802 kB 00:00 2024-02-18T11:45:24.793 INFO:teuthology.orchestra.run.smithi143.stdout:(95/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 194 kB 00:00 2024-02-18T11:45:24.818 INFO:teuthology.orchestra.run.smithi143.stdout:(96/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-18T11:45:24.844 INFO:teuthology.orchestra.run.smithi143.stdout:(97/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-18T11:45:24.877 INFO:teuthology.orchestra.run.smithi143.stdout:(98/176): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-02-18T11:45:24.902 INFO:teuthology.orchestra.run.smithi143.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-02-18T11:45:24.928 INFO:teuthology.orchestra.run.smithi143.stdout:(100/176): python3-pycparser-2.20-6.el9.noarch. 2.4 MB/s | 124 kB 00:00 2024-02-18T11:45:24.953 INFO:teuthology.orchestra.run.smithi143.stdout:(101/176): python3-requests-2.25.1-8.el9.noarch 2.2 MB/s | 113 kB 00:00 2024-02-18T11:45:24.978 INFO:teuthology.orchestra.run.smithi143.stdout:(102/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-18T11:45:25.107 INFO:teuthology.orchestra.run.smithi116.stdout:(18/176): ceph-osd-19.0.0-1555.g1c920afa.el9.x8 4.2 MB/s | 17 MB 00:04 2024-02-18T11:45:25.174 INFO:teuthology.orchestra.run.smithi116.stdout:(19/176): python3-ceph-argparse-19.0.0-1555.g1c 678 kB/s | 45 kB 00:00 2024-02-18T11:45:25.240 INFO:teuthology.orchestra.run.smithi116.stdout:(20/176): python3-ceph-common-19.0.0-1555.g1c92 1.9 MB/s | 131 kB 00:00 2024-02-18T11:45:25.307 INFO:teuthology.orchestra.run.smithi116.stdout:(21/176): librgw2-19.0.0-1555.g1c920afa.el9.x86 5.2 MB/s | 5.1 MB 00:00 2024-02-18T11:45:25.332 INFO:teuthology.orchestra.run.smithi116.stdout:(22/176): python3-cephfs-19.0.0-1555.g1c920afa. 1.7 MB/s | 161 kB 00:00 2024-02-18T11:45:25.391 INFO:teuthology.orchestra.run.smithi116.stdout:(23/176): python3-rados-19.0.0-1555.g1c920afa.e 3.7 MB/s | 320 kB 00:00 2024-02-18T11:45:25.416 INFO:teuthology.orchestra.run.smithi116.stdout:(24/176): python3-rbd-19.0.0-1555.g1c920afa.el9 3.5 MB/s | 298 kB 00:00 2024-02-18T11:45:25.442 INFO:teuthology.orchestra.run.smithi116.stdout:(25/176): python3-rgw-19.0.0-1555.g1c920afa.el9 1.9 MB/s | 99 kB 00:00 2024-02-18T11:45:25.445 INFO:teuthology.orchestra.run.smithi143.stdout:(103/176): ceph-test-19.0.0-1555.g1c920afa.el9. 7.6 MB/s | 47 MB 00:06 2024-02-18T11:45:25.477 INFO:teuthology.orchestra.run.smithi116.stdout:(26/176): rbd-fuse-19.0.0-1555.g1c920afa.el9.x8 1.4 MB/s | 85 kB 00:00 2024-02-18T11:45:25.534 INFO:teuthology.orchestra.run.smithi116.stdout:(27/176): rbd-nbd-19.0.0-1555.g1c920afa.el9.x86 2.9 MB/s | 171 kB 00:00 2024-02-18T11:45:25.593 INFO:teuthology.orchestra.run.smithi116.stdout:(28/176): ceph-grafana-dashboards-19.0.0-1555.g 539 kB/s | 31 kB 00:00 2024-02-18T11:45:25.660 INFO:teuthology.orchestra.run.smithi116.stdout:(29/176): ceph-mgr-cephadm-19.0.0-1555.g1c920af 2.0 MB/s | 138 kB 00:00 2024-02-18T11:45:25.935 INFO:teuthology.orchestra.run.smithi116.stdout:(30/176): rbd-mirror-19.0.0-1555.g1c920afa.el9. 6.3 MB/s | 3.1 MB 00:00 2024-02-18T11:45:25.977 INFO:teuthology.orchestra.run.smithi116.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1555.g1c920 5.6 MB/s | 1.8 MB 00:00 2024-02-18T11:45:26.004 INFO:teuthology.orchestra.run.smithi143.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 234 kB/s | 245 kB 00:01 2024-02-18T11:45:26.029 INFO:teuthology.orchestra.run.smithi143.stdout:(105/176): lua-devel-5.4.4-4.el9.x86_64.rpm 21 kB/s | 22 kB 00:01 2024-02-18T11:45:26.060 INFO:teuthology.orchestra.run.smithi116.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1555.g1c 2.9 MB/s | 248 kB 00:00 2024-02-18T11:45:26.127 INFO:teuthology.orchestra.run.smithi116.stdout:(33/176): ceph-mgr-rook-19.0.0-1555.g1c920afa.e 785 kB/s | 52 kB 00:00 2024-02-18T11:45:26.186 INFO:teuthology.orchestra.run.smithi116.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1555.g1 262 kB/s | 15 kB 00:00 2024-02-18T11:45:26.253 INFO:teuthology.orchestra.run.smithi116.stdout:(35/176): ceph-volume-19.0.0-1555.g1c920afa.el9 3.9 MB/s | 264 kB 00:00 2024-02-18T11:45:26.411 INFO:teuthology.orchestra.run.smithi116.stdout:(36/176): cephadm-19.0.0-1555.g1c920afa.el9.noa 4.6 MB/s | 748 kB 00:00 2024-02-18T11:45:26.638 INFO:teuthology.orchestra.run.smithi143.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 508 kB/s | 309 kB 00:00 2024-02-18T11:45:26.746 INFO:teuthology.orchestra.run.smithi143.stdout:(107/176): protobuf-compiler-3.14.0-13.el9.x86_ 664 kB/s | 863 kB 00:01 2024-02-18T11:45:26.771 INFO:teuthology.orchestra.run.smithi143.stdout:(108/176): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-02-18T11:45:26.797 INFO:teuthology.orchestra.run.smithi143.stdout:(109/176): abseil-cpp-20211102.0-3.el9.x86_64.r 692 kB/s | 548 kB 00:00 2024-02-18T11:45:26.812 INFO:teuthology.orchestra.run.smithi116.stdout:(37/176): ceph-test-19.0.0-1555.g1c920afa.el9.x 13 MB/s | 47 MB 00:03 2024-02-18T11:45:26.837 INFO:teuthology.orchestra.run.smithi116.stdout:(38/176): cephfs-top-19.0.0-1555.g1c920afa.el9. 43 kB/s | 18 kB 00:00 2024-02-18T11:45:26.872 INFO:teuthology.orchestra.run.smithi143.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-02-18T11:45:26.914 INFO:teuthology.orchestra.run.smithi143.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-02-18T11:45:26.971 INFO:teuthology.orchestra.run.smithi116.stdout:(39/176): automake-1.16.2-8.el9.noarch.rpm 4.1 MB/s | 662 kB 00:00 2024-02-18T11:45:26.981 INFO:teuthology.orchestra.run.smithi143.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 620 kB/s | 67 kB 00:00 2024-02-18T11:45:27.021 INFO:teuthology.orchestra.run.smithi116.stdout:(40/176): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2024-02-18T11:45:27.031 INFO:teuthology.orchestra.run.smithi143.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-02-18T11:45:27.046 INFO:teuthology.orchestra.run.smithi116.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-02-18T11:45:27.063 INFO:teuthology.orchestra.run.smithi116.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 491 kB/s | 20 kB 00:00 2024-02-18T11:45:27.088 INFO:teuthology.orchestra.run.smithi116.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 7.1 MB/s | 307 kB 00:00 2024-02-18T11:45:27.114 INFO:teuthology.orchestra.run.smithi116.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 632 kB/s | 32 kB 00:00 2024-02-18T11:45:27.198 INFO:teuthology.orchestra.run.smithi143.stdout:(114/176): python3-asyncssh-2.13.2-4.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-02-18T11:45:27.206 INFO:teuthology.orchestra.run.smithi116.stdout:(45/176): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-02-18T11:45:27.231 INFO:teuthology.orchestra.run.smithi143.stdout:(115/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.3 MB/s | 838 kB 00:00 2024-02-18T11:45:27.265 INFO:teuthology.orchestra.run.smithi116.stdout:(46/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-02-18T11:45:27.290 INFO:teuthology.orchestra.run.smithi116.stdout:(47/176): flexiblas-openblas-openmp-3.0.4-8.el9 98 kB/s | 17 kB 00:00 2024-02-18T11:45:27.290 INFO:teuthology.orchestra.run.smithi143.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-02-18T11:45:27.315 INFO:teuthology.orchestra.run.smithi116.stdout:(48/176): libaio-devel-0.3.111-13.el9.x86_64.rp 94 kB/s | 10 kB 00:00 2024-02-18T11:45:27.324 INFO:teuthology.orchestra.run.smithi143.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-02-18T11:45:27.340 INFO:teuthology.orchestra.run.smithi116.stdout:(49/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-02-18T11:45:27.365 INFO:teuthology.orchestra.run.smithi116.stdout:(50/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-02-18T11:45:27.432 INFO:teuthology.orchestra.run.smithi143.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-02-18T11:45:27.432 INFO:teuthology.orchestra.run.smithi116.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-02-18T11:45:27.432 INFO:teuthology.orchestra.run.smithi116.stdout:(52/176): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2024-02-18T11:45:27.441 INFO:teuthology.orchestra.run.smithi143.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-02-18T11:45:27.441 INFO:teuthology.orchestra.run.smithi116.stdout:(53/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 494 kB/s | 24 kB 00:00 2024-02-18T11:45:27.466 INFO:teuthology.orchestra.run.smithi116.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 623 kB/s | 46 kB 00:00 2024-02-18T11:45:27.491 INFO:teuthology.orchestra.run.smithi116.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-02-18T11:45:27.516 INFO:teuthology.orchestra.run.smithi116.stdout:(56/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-18T11:45:27.533 INFO:teuthology.orchestra.run.smithi143.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-02-18T11:45:27.542 INFO:teuthology.orchestra.run.smithi116.stdout:(57/176): openblas-0.3.21-2.el9.x86_64.rpm 457 kB/s | 34 kB 00:00 2024-02-18T11:45:27.558 INFO:teuthology.orchestra.run.smithi143.stdout:(121/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-02-18T11:45:27.634 INFO:teuthology.orchestra.run.smithi143.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-02-18T11:45:27.667 INFO:teuthology.orchestra.run.smithi143.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-02-18T11:45:27.684 INFO:teuthology.orchestra.run.smithi116.stdout:(58/176): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-02-18T11:45:27.743 INFO:teuthology.orchestra.run.smithi116.stdout:(59/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-02-18T11:45:27.751 INFO:teuthology.orchestra.run.smithi143.stdout:(124/176): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2024-02-18T11:45:27.776 INFO:teuthology.orchestra.run.smithi143.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-02-18T11:45:27.776 INFO:teuthology.orchestra.run.smithi116.stdout:(60/176): postgresql-13.11-1.el9.x86_64.rpm 6.6 MB/s | 1.5 MB 00:00 2024-02-18T11:45:27.801 INFO:teuthology.orchestra.run.smithi116.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 6.9 MB/s | 811 kB 00:00 2024-02-18T11:45:27.826 INFO:teuthology.orchestra.run.smithi116.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 134 kB 00:00 2024-02-18T11:45:27.851 INFO:teuthology.orchestra.run.smithi143.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-02-18T11:45:27.876 INFO:teuthology.orchestra.run.smithi143.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-02-18T11:45:27.902 INFO:teuthology.orchestra.run.smithi116.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-02-18T11:45:27.935 INFO:teuthology.orchestra.run.smithi143.stdout:(128/176): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-02-18T11:45:27.960 INFO:teuthology.orchestra.run.smithi143.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-02-18T11:45:27.969 INFO:teuthology.orchestra.run.smithi116.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-02-18T11:45:27.986 INFO:teuthology.orchestra.run.smithi143.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-02-18T11:45:27.994 INFO:teuthology.orchestra.run.smithi116.stdout:(65/176): python3-attrs-20.3.0-7.el9.noarch.rpm 472 kB/s | 78 kB 00:00 2024-02-18T11:45:28.035 INFO:teuthology.orchestra.run.smithi143.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-02-18T11:45:28.086 INFO:teuthology.orchestra.run.smithi143.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-02-18T11:45:28.094 INFO:teuthology.orchestra.run.smithi116.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-02-18T11:45:28.111 INFO:teuthology.orchestra.run.smithi143.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-02-18T11:45:28.119 INFO:teuthology.orchestra.run.smithi116.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 227 kB 00:00 2024-02-18T11:45:28.137 INFO:teuthology.orchestra.run.smithi143.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-02-18T11:45:28.144 INFO:teuthology.orchestra.run.smithi116.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 165 kB/s | 24 kB 00:00 2024-02-18T11:45:28.170 INFO:teuthology.orchestra.run.smithi116.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 242 kB/s | 18 kB 00:00 2024-02-18T11:45:28.187 INFO:teuthology.orchestra.run.smithi143.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 575 kB/s | 58 kB 00:00 2024-02-18T11:45:28.195 INFO:teuthology.orchestra.run.smithi116.stdout:(70/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-18T11:45:28.220 INFO:teuthology.orchestra.run.smithi116.stdout:(71/176): python3-markupsafe-1.1.1-12.el9.x86_6 421 kB/s | 32 kB 00:00 2024-02-18T11:45:28.228 INFO:teuthology.orchestra.run.smithi143.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-02-18T11:45:28.254 INFO:teuthology.orchestra.run.smithi143.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-02-18T11:45:28.288 INFO:teuthology.orchestra.run.smithi143.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-02-18T11:45:28.295 INFO:teuthology.orchestra.run.smithi116.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 4.8 MB 00:00 2024-02-18T11:45:28.320 INFO:teuthology.orchestra.run.smithi116.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 217 kB 00:00 2024-02-18T11:45:28.329 INFO:teuthology.orchestra.run.smithi143.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-02-18T11:45:28.346 INFO:teuthology.orchestra.run.smithi116.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 174 kB 00:00 2024-02-18T11:45:28.354 INFO:teuthology.orchestra.run.smithi143.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-02-18T11:45:28.371 INFO:teuthology.orchestra.run.smithi116.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 920 kB/s | 69 kB 00:00 2024-02-18T11:45:28.388 INFO:teuthology.orchestra.run.smithi143.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-02-18T11:45:28.396 INFO:teuthology.orchestra.run.smithi116.stdout:(76/176): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-02-18T11:45:28.421 INFO:teuthology.orchestra.run.smithi116.stdout:(77/176): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-02-18T11:45:28.430 INFO:teuthology.orchestra.run.smithi143.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 589 kB/s | 59 kB 00:00 2024-02-18T11:45:28.447 INFO:teuthology.orchestra.run.smithi116.stdout:(78/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-02-18T11:45:28.455 INFO:teuthology.orchestra.run.smithi143.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-02-18T11:45:28.472 INFO:teuthology.orchestra.run.smithi116.stdout:(79/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-18T11:45:28.489 INFO:teuthology.orchestra.run.smithi143.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2024-02-18T11:45:28.497 INFO:teuthology.orchestra.run.smithi116.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 628 kB/s | 47 kB 00:00 2024-02-18T11:45:28.539 INFO:teuthology.orchestra.run.smithi116.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 468 kB/s | 43 kB 00:00 2024-02-18T11:45:28.539 INFO:teuthology.orchestra.run.smithi143.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-02-18T11:45:28.556 INFO:teuthology.orchestra.run.smithi143.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-02-18T11:45:28.581 INFO:teuthology.orchestra.run.smithi116.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 464 kB/s | 39 kB 00:00 2024-02-18T11:45:28.598 INFO:teuthology.orchestra.run.smithi143.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-02-18T11:45:28.640 INFO:teuthology.orchestra.run.smithi143.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-02-18T11:45:28.657 INFO:teuthology.orchestra.run.smithi143.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-18T11:45:28.698 INFO:teuthology.orchestra.run.smithi116.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 262 kB/s | 41 kB 00:00 2024-02-18T11:45:28.732 INFO:teuthology.orchestra.run.smithi143.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-02-18T11:45:28.766 INFO:teuthology.orchestra.run.smithi143.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-18T11:45:28.833 INFO:teuthology.orchestra.run.smithi143.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 34 MB/s | 2.3 MB 00:00 2024-02-18T11:45:28.848 INFO:teuthology.orchestra.run.smithi116.stdout:(84/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-02-18T11:45:28.874 INFO:teuthology.orchestra.run.smithi116.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 745 kB/s | 215 kB 00:00 2024-02-18T11:45:28.890 INFO:teuthology.orchestra.run.smithi116.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 177 kB/s | 34 kB 00:00 2024-02-18T11:45:28.899 INFO:teuthology.orchestra.run.smithi143.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-02-18T11:45:28.907 INFO:teuthology.orchestra.run.smithi116.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.3 MB/s | 43 kB 00:00 2024-02-18T11:45:28.932 INFO:teuthology.orchestra.run.smithi116.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-02-18T11:45:28.933 INFO:teuthology.orchestra.run.smithi143.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-02-18T11:45:28.950 INFO:teuthology.orchestra.run.smithi116.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 970 kB/s | 56 kB 00:00 2024-02-18T11:45:28.967 INFO:teuthology.orchestra.run.smithi143.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 7.3 MB/s | 246 kB 00:00 2024-02-18T11:45:28.975 INFO:teuthology.orchestra.run.smithi116.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2024-02-18T11:45:29.000 INFO:teuthology.orchestra.run.smithi116.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 937 kB/s | 63 kB 00:00 2024-02-18T11:45:29.000 INFO:teuthology.orchestra.run.smithi143.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-02-18T11:45:29.017 INFO:teuthology.orchestra.run.smithi116.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 659 kB/s | 44 kB 00:00 2024-02-18T11:45:29.034 INFO:teuthology.orchestra.run.smithi143.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.0 MB/s | 35 kB 00:00 2024-02-18T11:45:29.042 INFO:teuthology.orchestra.run.smithi116.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2024-02-18T11:45:29.067 INFO:teuthology.orchestra.run.smithi116.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 2.7 MB/s | 186 kB 00:00 2024-02-18T11:45:29.084 INFO:teuthology.orchestra.run.smithi143.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-02-18T11:45:29.109 INFO:teuthology.orchestra.run.smithi116.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 802 kB 00:00 2024-02-18T11:45:29.126 INFO:teuthology.orchestra.run.smithi143.stdout:(159/176): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-02-18T11:45:29.134 INFO:teuthology.orchestra.run.smithi116.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 194 kB 00:00 2024-02-18T11:45:29.160 INFO:teuthology.orchestra.run.smithi116.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-02-18T11:45:29.185 INFO:teuthology.orchestra.run.smithi143.stdout:(160/176): iproute-6.7.0-2.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-02-18T11:45:29.185 INFO:teuthology.orchestra.run.smithi116.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-02-18T11:45:29.227 INFO:teuthology.orchestra.run.smithi116.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-02-18T11:45:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:(161/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 6.4 MB/s | 433 kB 00:00 2024-02-18T11:45:29.252 INFO:teuthology.orchestra.run.smithi116.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-02-18T11:45:29.277 INFO:teuthology.orchestra.run.smithi116.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.3 MB/s | 124 kB 00:00 2024-02-18T11:45:29.294 INFO:teuthology.orchestra.run.smithi143.stdout:(162/176): libacl-2.3.1-4.el9.x86_64.rpm 529 kB/s | 22 kB 00:00 2024-02-18T11:45:29.302 INFO:teuthology.orchestra.run.smithi116.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.5 MB/s | 113 kB 00:00 2024-02-18T11:45:29.328 INFO:teuthology.orchestra.run.smithi116.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-18T11:45:29.344 INFO:teuthology.orchestra.run.smithi143.stdout:(163/176): librados2-19.0.0-1555.g1c920afa.el9. 4.8 MB/s | 3.4 MB 00:00 2024-02-18T11:45:29.369 INFO:teuthology.orchestra.run.smithi143.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-02-18T11:45:29.394 INFO:teuthology.orchestra.run.smithi143.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-18T11:45:29.420 INFO:teuthology.orchestra.run.smithi143.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-02-18T11:45:29.445 INFO:teuthology.orchestra.run.smithi143.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-18T11:45:29.470 INFO:teuthology.orchestra.run.smithi143.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 553 kB/s | 28 kB 00:00 2024-02-18T11:45:29.504 INFO:teuthology.orchestra.run.smithi143.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-02-18T11:45:29.545 INFO:teuthology.orchestra.run.smithi143.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 29 MB/s | 2.1 MB 00:00 2024-02-18T11:45:29.604 INFO:teuthology.orchestra.run.smithi143.stdout:(171/176): librbd1-19.0.0-1555.g1c920afa.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-02-18T11:45:29.646 INFO:teuthology.orchestra.run.smithi143.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 16 MB/s | 2.2 MB 00:00 2024-02-18T11:45:29.678 INFO:teuthology.orchestra.run.smithi116.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 613 kB/s | 245 kB 00:00 2024-02-18T11:45:29.679 INFO:teuthology.orchestra.run.smithi143.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 3.3 MB/s | 438 kB 00:00 2024-02-18T11:45:29.705 INFO:teuthology.orchestra.run.smithi143.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 523 kB/s | 30 kB 00:00 2024-02-18T11:45:29.730 INFO:teuthology.orchestra.run.smithi143.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-02-18T11:45:29.763 INFO:teuthology.orchestra.run.smithi143.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-02-18T11:45:29.772 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:45:29.772 INFO:teuthology.orchestra.run.smithi143.stdout:Total 17 MB/s | 223 MB 00:13 2024-02-18T11:45:29.823 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-02-18T11:45:29.978 INFO:teuthology.orchestra.run.smithi116.stdout:(105/176): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-02-18T11:45:30.012 INFO:teuthology.orchestra.run.smithi116.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-02-18T11:45:30.062 INFO:teuthology.orchestra.run.smithi116.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-02-18T11:45:30.124 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-02-18T11:45:30.124 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-02-18T11:45:30.321 INFO:teuthology.orchestra.run.smithi116.stdout:(108/176): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-02-18T11:45:30.346 INFO:teuthology.orchestra.run.smithi116.stdout:(109/176): gperftools-libs-2.9.1-2.el9.x86_64.r 841 kB/s | 309 kB 00:00 2024-02-18T11:45:30.380 INFO:teuthology.orchestra.run.smithi116.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-02-18T11:45:30.413 INFO:teuthology.orchestra.run.smithi116.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 725 kB/s | 49 kB 00:00 2024-02-18T11:45:30.439 INFO:teuthology.orchestra.run.smithi116.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-02-18T11:45:30.497 INFO:teuthology.orchestra.run.smithi116.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-02-18T11:45:30.531 INFO:teuthology.orchestra.run.smithi116.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-02-18T11:45:30.564 INFO:teuthology.orchestra.run.smithi116.stdout:(115/176): protobuf-compiler-3.14.0-13.el9.x86_ 698 kB/s | 863 kB 00:01 2024-02-18T11:45:30.686 INFO:teuthology.orchestra.run.smithi116.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 281 kB/s | 43 kB 00:00 2024-02-18T11:45:30.724 INFO:teuthology.orchestra.run.smithi116.stdout:(117/176): python3-asyncssh-2.13.2-4.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-02-18T11:45:30.757 INFO:teuthology.orchestra.run.smithi116.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 211 kB/s | 15 kB 00:00 2024-02-18T11:45:30.824 INFO:teuthology.orchestra.run.smithi116.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-02-18T11:45:30.849 INFO:teuthology.orchestra.run.smithi116.stdout:(120/176): python3-cachetools-4.2.4-1.el9.noarc 113 kB/s | 32 kB 00:00 2024-02-18T11:45:30.875 INFO:teuthology.orchestra.run.smithi116.stdout:(121/176): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-02-18T11:45:30.941 INFO:teuthology.orchestra.run.smithi116.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-02-18T11:45:30.967 INFO:teuthology.orchestra.run.smithi116.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-02-18T11:45:30.992 INFO:teuthology.orchestra.run.smithi116.stdout:(124/176): python3-colorama-0.4.4-12.el9.noarch 248 kB/s | 35 kB 00:00 2024-02-18T11:45:31.042 INFO:teuthology.orchestra.run.smithi116.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-02-18T11:45:31.068 INFO:teuthology.orchestra.run.smithi116.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-02-18T11:45:31.110 INFO:teuthology.orchestra.run.smithi116.stdout:(127/176): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-02-18T11:45:31.160 INFO:teuthology.orchestra.run.smithi116.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-02-18T11:45:31.193 INFO:teuthology.orchestra.run.smithi116.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-02-18T11:45:31.235 INFO:teuthology.orchestra.run.smithi116.stdout:(130/176): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-02-18T11:45:31.261 INFO:teuthology.orchestra.run.smithi116.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-02-18T11:45:31.303 INFO:teuthology.orchestra.run.smithi116.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-02-18T11:45:31.328 INFO:teuthology.orchestra.run.smithi116.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-02-18T11:45:31.387 INFO:teuthology.orchestra.run.smithi116.stdout:(134/176): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-02-18T11:45:31.412 INFO:teuthology.orchestra.run.smithi116.stdout:(135/176): python3-more-itertools-8.12.0-2.el9. 522 kB/s | 79 kB 00:00 2024-02-18T11:45:31.437 INFO:teuthology.orchestra.run.smithi116.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-02-18T11:45:31.462 INFO:teuthology.orchestra.run.smithi116.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-02-18T11:45:31.496 INFO:teuthology.orchestra.run.smithi116.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-02-18T11:45:31.521 INFO:teuthology.orchestra.run.smithi116.stdout:(139/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-02-18T11:45:31.546 INFO:teuthology.orchestra.run.smithi116.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-02-18T11:45:31.572 INFO:teuthology.orchestra.run.smithi116.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-02-18T11:45:31.597 INFO:teuthology.orchestra.run.smithi116.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 780 kB/s | 59 kB 00:00 2024-02-18T11:45:31.622 INFO:teuthology.orchestra.run.smithi116.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 472 kB/s | 36 kB 00:00 2024-02-18T11:45:31.648 INFO:teuthology.orchestra.run.smithi116.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-02-18T11:45:31.681 INFO:teuthology.orchestra.run.smithi116.stdout:(145/176): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-02-18T11:45:31.715 INFO:teuthology.orchestra.run.smithi116.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-02-18T11:45:31.740 INFO:teuthology.orchestra.run.smithi116.stdout:(147/176): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-02-18T11:45:31.766 INFO:teuthology.orchestra.run.smithi116.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-02-18T11:45:31.791 INFO:teuthology.orchestra.run.smithi116.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-02-18T11:45:31.916 INFO:teuthology.orchestra.run.smithi116.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-02-18T11:45:31.949 INFO:teuthology.orchestra.run.smithi116.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-18T11:45:32.017 INFO:teuthology.orchestra.run.smithi116.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 34 MB/s | 2.3 MB 00:00 2024-02-18T11:45:32.075 INFO:teuthology.orchestra.run.smithi116.stdout:(153/176): NetworkManager-libnm-1.45.91-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-02-18T11:45:32.088 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-02-18T11:45:32.088 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-02-18T11:45:32.100 INFO:teuthology.orchestra.run.smithi116.stdout:(154/176): NetworkManager-team-1.45.91-1.el9.x8 1.6 MB/s | 41 kB 00:00 2024-02-18T11:45:32.134 INFO:teuthology.orchestra.run.smithi116.stdout:(155/176): NetworkManager-tui-1.45.91-1.el9.x86 7.2 MB/s | 246 kB 00:00 2024-02-18T11:45:32.167 INFO:teuthology.orchestra.run.smithi116.stdout:(156/176): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2024-02-18T11:45:32.193 INFO:teuthology.orchestra.run.smithi116.stdout:(157/176): elfutils-debuginfod-client-0.190-2.e 1.4 MB/s | 35 kB 00:00 2024-02-18T11:45:32.226 INFO:teuthology.orchestra.run.smithi116.stdout:(158/176): elfutils-libelf-0.190-2.el9.x86_64.r 5.6 MB/s | 191 kB 00:00 2024-02-18T11:45:32.285 INFO:teuthology.orchestra.run.smithi116.stdout:(159/176): librados2-19.0.0-1555.g1c920afa.el9. 6.5 MB/s | 3.4 MB 00:00 2024-02-18T11:45:32.310 INFO:teuthology.orchestra.run.smithi116.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 254 kB 00:00 2024-02-18T11:45:32.344 INFO:teuthology.orchestra.run.smithi116.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 14 MB/s | 802 kB 00:00 2024-02-18T11:45:32.378 INFO:teuthology.orchestra.run.smithi116.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 6.5 MB/s | 433 kB 00:00 2024-02-18T11:45:32.394 INFO:teuthology.orchestra.run.smithi116.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-02-18T11:45:32.420 INFO:teuthology.orchestra.run.smithi116.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 107 kB 00:00 2024-02-18T11:45:32.445 INFO:teuthology.orchestra.run.smithi116.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-18T11:45:32.470 INFO:teuthology.orchestra.run.smithi116.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-02-18T11:45:32.487 INFO:teuthology.orchestra.run.smithi116.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.5 MB/s | 63 kB 00:00 2024-02-18T11:45:32.504 INFO:teuthology.orchestra.run.smithi116.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 830 kB/s | 28 kB 00:00 2024-02-18T11:45:32.538 INFO:teuthology.orchestra.run.smithi116.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 23 MB/s | 1.2 MB 00:00 2024-02-18T11:45:32.588 INFO:teuthology.orchestra.run.smithi116.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 26 MB/s | 2.1 MB 00:00 2024-02-18T11:45:32.630 INFO:teuthology.orchestra.run.smithi116.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 25 MB/s | 2.2 MB 00:00 2024-02-18T11:45:32.656 INFO:teuthology.orchestra.run.smithi116.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 6.6 MB/s | 438 kB 00:00 2024-02-18T11:45:32.674 INFO:teuthology.orchestra.run.smithi116.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 732 kB/s | 30 kB 00:00 2024-02-18T11:45:32.698 INFO:teuthology.orchestra.run.smithi116.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-02-18T11:45:32.732 INFO:teuthology.orchestra.run.smithi116.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-02-18T11:45:32.790 INFO:teuthology.orchestra.run.smithi116.stdout:(176/176): librbd1-19.0.0-1555.g1c920afa.el9.x8 3.1 MB/s | 3.1 MB 00:00 2024-02-18T11:45:32.795 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:45:32.796 INFO:teuthology.orchestra.run.smithi116.stdout:Total 16 MB/s | 223 MB 00:14 2024-02-18T11:45:32.846 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-02-18T11:45:33.141 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-02-18T11:45:33.141 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-02-18T11:45:35.076 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-02-18T11:45:35.077 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-02-18T11:45:39.039 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-02-18T11:45:39.129 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-18T11:45:39.161 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-18T11:45:39.199 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:45:39.225 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:45:39.286 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-18T11:45:39.324 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-18T11:45:39.385 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-18T11:45:39.419 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-18T11:45:39.480 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-18T11:45:39.754 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-18T11:45:40.083 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-18T11:45:40.143 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:45:40.272 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:45:40.320 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:45:40.395 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:45:40.433 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:45:40.600 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-18T11:45:40.654 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-18T11:45:40.682 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:45:40.768 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:45:40.791 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:45:40.840 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:45:40.878 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-18T11:45:40.901 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:45:40.917 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:45:41.101 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:45:41.132 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:45:41.172 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-18T11:45:41.361 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-18T11:45:41.519 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:45:41.820 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:45:41.951 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:45:42.017 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:45:42.253 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:45:42.264 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-02-18T11:45:42.360 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-18T11:45:42.378 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-18T11:45:42.395 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-18T11:45:42.442 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:45:42.469 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-18T11:45:42.476 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-18T11:45:42.506 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-18T11:45:42.512 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-18T11:45:42.550 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-18T11:45:42.574 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-18T11:45:42.607 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-18T11:45:42.637 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-18T11:45:42.646 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-18T11:45:42.667 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-18T11:45:42.672 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-18T11:45:42.702 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-18T11:45:42.725 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-18T11:45:42.742 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-18T11:45:42.806 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-18T11:45:42.841 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-18T11:45:42.885 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-18T11:45:42.917 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-18T11:45:42.945 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-18T11:45:43.014 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-18T11:45:43.050 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 39/202 2024-02-18T11:45:43.059 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-18T11:45:43.085 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 40/202 2024-02-18T11:45:43.113 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 41/202 2024-02-18T11:45:43.114 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:45:43.182 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-18T11:45:43.220 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:45:43.228 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-18T11:45:43.249 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:45:43.301 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-18T11:45:43.339 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:45:43.376 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:45:43.437 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-18T11:45:43.488 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-18T11:45:43.528 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-18T11:45:43.575 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-18T11:45:43.603 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:45:43.689 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:45:43.712 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:45:43.774 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 17/202 2024-02-18T11:45:43.820 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-18T11:45:43.844 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:45:43.859 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-18T11:45:44.002 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:45:44.015 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-18T11:45:44.034 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-18T11:45:44.073 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-18T11:45:44.092 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-18T11:45:44.173 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-18T11:45:44.214 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:45:44.222 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:45:44.261 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-18T11:45:44.262 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-18T11:45:44.298 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 52/202 2024-02-18T11:45:44.340 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-18T11:45:44.400 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:45:44.450 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-18T11:45:44.485 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-18T11:45:44.515 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-18T11:45:44.570 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:45:44.620 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:45:44.819 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-18T11:45:44.830 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-18T11:45:44.946 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-18T11:45:45.038 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-18T11:45:45.067 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-18T11:45:45.135 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-18T11:45:45.189 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-18T11:45:45.192 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-18T11:45:45.229 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-18T11:45:45.265 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-18T11:45:45.268 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-18T11:45:45.305 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-18T11:45:45.314 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-18T11:45:45.369 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-18T11:45:45.404 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-18T11:45:45.447 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-18T11:45:45.460 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-18T11:45:45.496 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-18T11:45:45.523 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-18T11:45:45.524 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-18T11:45:45.579 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-18T11:45:45.595 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-18T11:45:45.629 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 39/202 2024-02-18T11:45:45.651 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-18T11:45:45.663 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 40/202 2024-02-18T11:45:45.682 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-18T11:45:45.693 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 41/202 2024-02-18T11:45:45.719 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-18T11:45:45.761 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-18T11:45:45.766 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-18T11:45:45.801 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-18T11:45:45.807 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-18T11:45:45.844 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-18T11:45:45.880 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-18T11:45:45.887 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-18T11:45:46.011 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-18T11:45:46.021 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-18T11:45:46.059 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-18T11:45:46.076 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-18T11:45:46.106 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-18T11:45:46.148 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-18T11:45:46.243 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-18T11:45:46.281 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-18T11:45:46.321 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-18T11:45:46.347 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-18T11:45:46.371 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-18T11:45:46.459 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-18T11:45:46.576 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-18T11:45:46.579 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-18T11:45:46.656 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-18T11:45:46.681 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-18T11:45:46.730 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-18T11:45:46.736 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-18T11:45:46.778 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-18T11:45:46.783 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:45:46.792 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-18T11:45:46.830 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-18T11:45:46.835 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-18T11:45:46.859 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 52/202 2024-02-18T11:45:46.901 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-18T11:45:46.998 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-18T11:45:47.018 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-18T11:45:47.060 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-18T11:45:47.061 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-18T11:45:47.204 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-18T11:45:47.244 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-18T11:45:47.282 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-18T11:45:47.314 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-18T11:45:47.349 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-18T11:45:47.364 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-18T11:45:47.380 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-18T11:45:47.417 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-18T11:45:47.451 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-18T11:45:47.490 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-18T11:45:47.521 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-18T11:45:47.535 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-18T11:45:47.552 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-18T11:45:47.621 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-18T11:45:47.629 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-18T11:45:47.662 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-18T11:45:47.664 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-18T11:45:47.695 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-18T11:45:47.737 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-18T11:45:47.772 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-18T11:45:47.814 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-18T11:45:47.862 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-18T11:45:47.867 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-18T11:45:47.921 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-18T11:45:47.960 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-18T11:45:47.985 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-18T11:45:48.002 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-18T11:45:48.034 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-18T11:45:48.081 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-18T11:45:48.097 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-18T11:45:48.179 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-18T11:45:48.228 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-18T11:45:48.272 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-18T11:45:48.319 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-18T11:45:48.354 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-18T11:45:48.397 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-18T11:45:48.440 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-18T11:45:48.572 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-18T11:45:48.619 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-18T11:45:48.650 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-18T11:45:48.691 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-18T11:45:48.785 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-18T11:45:48.825 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-18T11:45:48.873 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-18T11:45:48.900 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-18T11:45:48.924 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-18T11:45:48.948 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-18T11:45:48.991 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-18T11:45:49.000 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-18T11:45:49.026 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-18T11:45:49.078 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-18T11:45:49.132 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-18T11:45:49.167 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-18T11:45:49.214 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-18T11:45:49.229 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-18T11:45:49.273 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-18T11:45:49.322 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-18T11:45:49.390 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-18T11:45:49.403 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-18T11:45:49.421 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-18T11:45:49.499 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-18T11:45:49.594 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-18T11:45:49.853 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-18T11:45:50.108 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-18T11:45:50.133 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-18T11:45:50.151 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-18T11:45:50.183 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-18T11:45:50.224 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-18T11:45:50.252 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-18T11:45:50.259 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-18T11:45:50.289 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-18T11:45:50.318 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-18T11:45:50.360 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-18T11:45:50.400 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-18T11:45:50.431 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-18T11:45:50.461 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-18T11:45:50.537 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-18T11:45:50.572 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-18T11:45:50.605 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-18T11:45:50.646 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-18T11:45:50.681 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-18T11:45:50.769 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-18T11:45:50.859 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-18T11:45:50.903 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-18T11:45:50.935 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-18T11:45:50.982 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-18T11:45:51.080 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-18T11:45:51.137 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-18T11:45:51.466 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-18T11:45:51.542 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-18T11:45:51.569 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-18T11:45:51.859 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-18T11:45:51.882 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-18T11:45:51.908 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-18T11:45:51.915 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:45:51.942 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-18T11:45:51.990 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-18T11:45:51.997 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:45:52.029 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 126/202 2024-02-18T11:45:52.074 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-18T11:45:52.078 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-18T11:45:52.118 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-18T11:45:52.295 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-18T11:45:52.324 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-18T11:45:52.401 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-18T11:45:52.535 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-18T11:45:52.561 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:45:52.632 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:45:52.662 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 130/202 2024-02-18T11:45:53.031 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-18T11:45:53.137 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-18T11:45:54.357 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-18T11:45:54.436 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-18T11:45:54.463 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-18T11:45:54.772 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-18T11:45:54.802 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:45:54.873 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:45:54.874 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 125/202 2024-02-18T11:45:54.906 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 126/202 2024-02-18T11:45:54.909 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:45:54.943 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-18T11:45:55.397 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-18T11:45:55.421 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:45:55.453 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:45:55.475 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 129/202 2024-02-18T11:45:55.488 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:45:55.506 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 130/202 2024-02-18T11:45:55.581 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:45:55.581 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-18T11:45:55.582 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-18T11:45:55.582 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:45:55.622 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:45:57.636 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:45:57.669 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:45:58.222 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 131/202 2024-02-18T11:45:58.255 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:45:58.330 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 132/202 2024-02-18T11:45:58.330 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-18T11:45:58.330 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-18T11:45:58.330 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:45:58.367 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-02-18T11:46:07.170 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:07.339 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:46:07.353 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:46:07.353 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:07.353 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-18T11:46:07.354 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:08.833 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:46:08.859 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:46:08.859 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:08.859 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-18T11:46:08.859 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:46:08.859 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:46:08.860 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:09.067 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:46:09.090 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:46:09.090 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:09.090 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-18T11:46:09.090 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:46:09.090 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:46:09.090 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:09.526 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:46:09.549 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:46:09.549 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:09.549 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-18T11:46:09.549 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:46:09.549 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:46:09.549 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:09.653 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-18T11:46:09.796 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 133/202 2024-02-18T11:46:09.796 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /sys 2024-02-18T11:46:09.796 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /proc 2024-02-18T11:46:09.797 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /mnt 2024-02-18T11:46:09.797 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /var/tmp 2024-02-18T11:46:09.797 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /home 2024-02-18T11:46:09.797 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /root 2024-02-18T11:46:09.797 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /tmp 2024-02-18T11:46:09.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:09.953 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:46:09.968 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 134/202 2024-02-18T11:46:09.968 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:09.968 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-18T11:46:09.968 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:10.104 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:46:10.134 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:46:10.166 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 140/202 2024-02-18T11:46:10.226 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 141/202 2024-02-18T11:46:10.249 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:46:10.479 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:46:10.509 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:46:11.448 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:46:11.473 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 135/202 2024-02-18T11:46:11.473 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:11.473 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-18T11:46:11.473 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:46:11.473 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-18T11:46:11.473 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:11.479 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:46:11.509 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:46:11.610 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:46:11.673 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:46:11.695 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 136/202 2024-02-18T11:46:11.695 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:11.695 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-18T11:46:11.695 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:46:11.696 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-18T11:46:11.696 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:11.764 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 145/202 2024-02-18T11:46:11.803 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:46:11.827 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:46:11.827 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:11.827 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-18T11:46:11.827 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:46:11.827 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:46:11.827 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:11.870 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:46:11.882 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:46:11.936 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 148/202 2024-02-18T11:46:12.132 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:46:12.154 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 137/202 2024-02-18T11:46:12.155 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:12.155 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-18T11:46:12.155 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:46:12.155 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-18T11:46:12.155 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:12.251 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-18T11:46:12.660 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:46:12.690 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 139/202 2024-02-18T11:46:12.722 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 140/202 2024-02-18T11:46:12.782 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 141/202 2024-02-18T11:46:12.790 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-18T11:46:12.805 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:46:12.822 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:46:12.845 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:46:12.846 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:12.846 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-18T11:46:12.846 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:46:12.846 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:46:12.846 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:12.890 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:46:12.912 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:46:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:12.912 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-18T11:46:12.912 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:12.957 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:46:12.979 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:46:12.979 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:12.980 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-18T11:46:12.980 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:46:12.980 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:46:12.980 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:13.033 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 142/202 2024-02-18T11:46:13.067 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:46:13.283 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:46:13.306 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:46:13.306 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:13.306 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-18T11:46:13.306 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:46:13.306 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:46:13.306 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:14.029 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 143/202 2024-02-18T11:46:14.057 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:46:14.157 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 144/202 2024-02-18T11:46:14.319 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 145/202 2024-02-18T11:46:14.351 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:46:14.373 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 146/202 2024-02-18T11:46:14.374 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:14.374 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-18T11:46:14.374 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:46:14.374 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-18T11:46:14.374 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:14.418 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:46:14.429 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 147/202 2024-02-18T11:46:14.479 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 148/202 2024-02-18T11:46:15.344 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-18T11:46:15.370 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:46:15.392 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 150/202 2024-02-18T11:46:15.392 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:15.392 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-18T11:46:15.392 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:46:15.393 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-18T11:46:15.393 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:15.445 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:46:15.467 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1555.g1c920 151/202 2024-02-18T11:46:15.467 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:15.467 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-18T11:46:15.467 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:15.513 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:46:15.535 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 152/202 2024-02-18T11:46:15.535 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:15.535 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-18T11:46:15.535 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:46:15.535 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-18T11:46:15.535 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:15.847 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:46:15.869 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:46:15.869 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-18T11:46:15.869 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-18T11:46:15.869 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:46:15.869 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-18T11:46:15.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:17.771 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 154/202 2024-02-18T11:46:17.812 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 155/202 2024-02-18T11:46:17.851 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 156/202 2024-02-18T11:46:17.887 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-18T11:46:18.078 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 158/202 2024-02-18T11:46:18.188 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-18T11:46:18.226 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 160/202 2024-02-18T11:46:18.280 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-18T11:46:18.335 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-18T11:46:18.409 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-18T11:46:18.454 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-18T11:46:18.497 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-18T11:46:18.528 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 166/202 2024-02-18T11:46:18.558 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-18T11:46:18.637 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-18T11:46:18.683 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-18T11:46:18.715 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-18T11:46:19.291 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:46:19.329 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:46:19.350 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:46:20.167 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 172/202 2024-02-18T11:46:20.227 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-18T11:46:20.268 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-18T11:46:20.301 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 154/202 2024-02-18T11:46:20.342 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 155/202 2024-02-18T11:46:20.360 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-18T11:46:20.372 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 156/202 2024-02-18T11:46:20.389 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-18T11:46:20.399 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-18T11:46:20.414 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-18T11:46:20.440 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-18T11:46:20.466 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-18T11:46:20.466 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:46:20.534 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:46:20.562 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-18T11:46:20.581 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 158/202 2024-02-18T11:46:20.592 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-18T11:46:20.592 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:46:20.678 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:46:20.691 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-18T11:46:20.704 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-18T11:46:20.731 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 160/202 2024-02-18T11:46:20.745 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-18T11:46:20.770 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-18T11:46:20.794 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-18T11:46:20.798 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-18T11:46:20.830 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-18T11:46:20.864 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-18T11:46:20.889 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-18T11:46:20.912 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-18T11:46:20.952 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-18T11:46:20.971 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-18T11:46:20.977 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-18T11:46:20.999 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-18T11:46:21.002 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-18T11:46:21.028 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:46:21.034 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-18T11:46:21.053 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-18T11:46:21.057 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 166/202 2024-02-18T11:46:21.060 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:46:21.060 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:46:21.086 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-18T11:46:21.157 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-18T11:46:21.205 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-18T11:46:21.235 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-18T11:46:21.248 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:46:21.248 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:46:21.331 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:46:21.355 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:46:21.381 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-18T11:46:21.406 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-18T11:46:21.439 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-18T11:46:21.506 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:46:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:46:21.891 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:46:21.910 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-18T11:46:22.724 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 172/202 2024-02-18T11:46:22.805 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-18T11:46:22.846 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-18T11:46:22.929 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-18T11:46:22.966 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-18T11:46:22.992 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-18T11:46:23.017 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-18T11:46:23.043 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-18T11:46:23.043 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:46:23.128 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-18T11:46:23.156 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-18T11:46:23.186 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-18T11:46:23.186 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:46:23.271 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-18T11:46:23.305 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-18T11:46:23.339 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-18T11:46:23.364 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-18T11:46:23.391 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-18T11:46:23.423 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-18T11:46:23.458 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-18T11:46:23.505 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-18T11:46:23.537 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-18T11:46:23.562 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-18T11:46:23.587 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-18T11:46:23.612 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:46:23.638 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-18T11:46:23.644 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:46:23.644 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:46:23.832 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-18T11:46:23.832 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:46:23.923 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-18T11:46:23.948 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:46:23.973 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-18T11:46:23.998 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-18T11:46:24.032 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-18T11:46:24.099 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:46:26.403 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:46:26.403 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 1/202 2024-02-18T11:46:26.403 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2/202 2024-02-18T11:46:26.403 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 3/202 2024-02-18T11:46:26.403 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 4/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 5/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 6/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 7/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 8/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 9/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 10/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 14/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 15/202 2024-02-18T11:46:26.404 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 17/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 18/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 19/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 20/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 21/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 22/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 23/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 24/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 25/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 26/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 27/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 28/202 2024-02-18T11:46:26.405 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 29/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 30/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 31/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 32/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 33/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 34/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 35/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 36/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 37/202 2024-02-18T11:46:26.422 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 38/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 39/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-18T11:46:26.423 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-18T11:46:26.424 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-18T11:46:26.425 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-18T11:46:26.426 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-18T11:46:26.427 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-18T11:46:26.428 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-18T11:46:26.429 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-18T11:46:26.430 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 151/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-18T11:46:26.431 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-18T11:46:26.432 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-18T11:46:26.433 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-18T11:46:26.434 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-18T11:46:28.874 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-18T11:46:28.874 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 1/202 2024-02-18T11:46:28.874 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2/202 2024-02-18T11:46:28.874 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 3/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 4/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1555.g1c920 5/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 6/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 7/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 8/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 9/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 10/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 11/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 12/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 13/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 14/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_ 15/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 16/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 17/202 2024-02-18T11:46:28.875 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_6 18/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86 19/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 20/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el 21/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-common-2:19.0.0-1555.g1c920afa.el9. 22/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_6 23/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 24/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 25/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 26/202 2024-02-18T11:46:28.876 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 27/202 2024-02-18T11:46:28.892 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 28/202 2024-02-18T11:46:28.892 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 29/202 2024-02-18T11:46:28.892 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa. 30/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noa 31/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.n 32/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c9 33/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el 34/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 35/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.e 36/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 37/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 38/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 39/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-18T11:46:28.893 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-18T11:46:28.894 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-18T11:46:28.895 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-18T11:46:28.896 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-18T11:46:28.897 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-18T11:46:28.898 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-18T11:46:28.899 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-18T11:46:28.900 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-18T11:46:28.901 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 151/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 153/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-18T11:46:28.902 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-18T11:46:28.903 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-18T11:46:28.904 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-18T11:46:28.905 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-18T11:46:29.197 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: acl-2.3.1-4.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-18T11:46:29.198 INFO:teuthology.orchestra.run.smithi143.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.199 INFO:teuthology.orchestra.run.smithi143.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.200 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.201 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: jq-1.6-16.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-18T11:46:29.202 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-18T11:46:29.203 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-18T11:46:29.204 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.11-1.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.205 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-18T11:46:29.206 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-18T11:46:29.207 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-18T11:46:29.208 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-18T11:46:29.209 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:29.210 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-3.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-18T11:46:29.211 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:29.212 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-02-18T11:46:29.803 DEBUG:teuthology.parallel:result is None 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout:Upgraded: 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: acl-2.3.1-4.el9.x86_64 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-18T11:46:31.594 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-18T11:46:31.595 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: automake-1.16.2-8.el9.noarch 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: bison-3.7.4-5.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.596 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.597 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top-2:19.0.0-1555.g1c920afa.el9.noarch 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: flex-2.6.4-9.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-18T11:46:31.598 INFO:teuthology.orchestra.run.smithi116.stdout: jq-1.6-16.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.599 INFO:teuthology.orchestra.run.smithi116.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: lua-5.4.4-4.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-13.11-1.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-18T11:46:31.600 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-18T11:46:31.601 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-18T11:46:31.602 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply-3.11-14.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-18T11:46:31.603 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-18T11:46:31.604 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-2:19.0.0-1555.g1c920afa.el9.x86_64 2024-02-18T11:46:31.605 INFO:teuthology.orchestra.run.smithi116.stdout: re2-1:20211101-3.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:31.606 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-02-18T11:46:32.217 DEBUG:teuthology.parallel:result is None 2024-02-18T11:46:32.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:46:32.359 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T11:46:32.382 INFO:teuthology.orchestra.run.smithi116.stdout:19.0.0-1555.g1c920afa.el9 2024-02-18T11:46:32.382 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1555.g1c920afa.el9 2024-02-18T11:46:32.382 INFO:teuthology.task.install:The correct ceph version 19.0.0-1555.g1c920afa is installed. 2024-02-18T11:46:32.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1c920afa8316ea21877f3186850254947531ea22 2024-02-18T11:46:32.606 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-18T11:46:32.631 INFO:teuthology.orchestra.run.smithi143.stdout:19.0.0-1555.g1c920afa.el9 2024-02-18T11:46:32.631 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1555.g1c920afa.el9 2024-02-18T11:46:32.631 INFO:teuthology.task.install:The correct ceph version 19.0.0-1555.g1c920afa is installed. 2024-02-18T11:46:32.633 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-18T11:46:32.633 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:32.633 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T11:46:32.664 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:32.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T11:46:32.705 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-18T11:46:32.705 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:32.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T11:46:32.734 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T11:46:32.802 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:32.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T11:46:32.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T11:46:32.900 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-18T11:46:32.900 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:32.900 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T11:46:32.927 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T11:46:32.994 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:32.994 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T11:46:33.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T11:46:33.091 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-18T11:46:33.091 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:33.091 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T11:46:33.118 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T11:46:33.186 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:33.186 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T11:46:33.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T11:46:33.282 INFO:teuthology.run_tasks:Running task ceph... 2024-02-18T11:46:33.363 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-02-18T11:46:33.363 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /var/log/ceph 2024-02-18T11:46:33.365 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2024-02-18T11:46:33.392 INFO:tasks.ceph:Disabling ceph logrotate... 2024-02-18T11:46:33.393 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-18T11:46:33.433 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2024-02-18T11:46:33.460 INFO:tasks.ceph:Creating extra log directories... 2024-02-18T11:46:33.460 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-18T11:46:33.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-02-18T11:46:33.534 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-02-18T11:46:33.534 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-02-18T11:46:33.534 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-02-18T11:46:33.534 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:33.534 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-18T11:46:33.580 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-18T11:46:33.698 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-18T11:46:33.725 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:33.725 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-02-18T11:46:33.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-02-18T11:46:33.860 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-02-18T11:46:33.891 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-02-18T11:46:33.892 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-02-18T11:46:33.892 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi/7564390', 'branch': 'wip-vshankar-testing-20240217.015652', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7564390', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8d59ab0bd93c806231018a6729893bf309624262'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-02-17_05:57:56-fs-wip-vshankar-testing-20240217.015652-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240217.015652'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1c920afa8316ea21877f3186850254947531ea22'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240217.015652', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 218, 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'sleep_before_teardown': 0, 'subset': '793/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240217.015652', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1c920afa8316ea21877f3186850254947531ea22/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1c920afa8316ea21877f3186850254947531ea22', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi116.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDF9hxeylZvSIybdPlGqIGIbFHyNqvDdbsuE5RWtwdB3vgNLRp5/nPhyR3wlP1EHgKAX/1YAkAH6Q8yNAcW5Wo4=', 'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDCibNCnHfCeF3XWLyecPEfPfJjG6P4jVBVwWVy1KdM1d0tEZbwJj+cQYNBOts2ON6zdY0t2gzEntEL3CWvXKVo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8d59ab0bd93c806231018a6729893bf309624262'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c920afa8316ea21877f3186850254947531ea22'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1c920afa8316ea21877f3186850254947531ea22', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-02-17_05:57:56', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-02-18T11:46:33.892 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-18T11:46:33.895 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-02-18T11:46:33.943 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/run/ceph 2024-02-18T11:46:33.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2024-02-18T11:46:34.009 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:34.009 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T11:46:34.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T11:46:34.029 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-02-18T11:46:34.083 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-02-18 11:46:24.434353861 +0000 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-02-18 11:44:31.240809472 +0000 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-02-18 11:44:31.240809472 +0000 2024-02-18T11:46:34.084 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-02-18 11:44:31.240809472 +0000 2024-02-18T11:46:34.084 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T11:46:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-02-18T11:46:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-02-18T11:46:34.149 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000177085 s, 2.9 MB/s 2024-02-18T11:46:34.151 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T11:46:34.207 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-02-18 11:46:24.435353844 +0000 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-02-18 11:44:31.584803732 +0000 2024-02-18T11:46:34.262 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-02-18 11:44:31.584803732 +0000 2024-02-18T11:46:34.263 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-02-18 11:44:31.584803732 +0000 2024-02-18T11:46:34.263 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T11:46:34.327 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-02-18T11:46:34.327 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-02-18T11:46:34.327 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000200335 s, 2.6 MB/s 2024-02-18T11:46:34.328 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T11:46:34.384 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-02-18T11:46:34.439 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-02-18 11:46:24.435353844 +0000 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-02-18 11:44:31.930797959 +0000 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-02-18 11:44:31.930797959 +0000 2024-02-18T11:46:34.440 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-02-18 11:44:31.930797959 +0000 2024-02-18T11:46:34.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T11:46:34.505 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-02-18T11:46:34.506 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-02-18T11:46:34.506 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000222743 s, 2.3 MB/s 2024-02-18T11:46:34.507 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T11:46:34.562 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-02-18T11:46:34.617 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T11:46:34.617 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:34.617 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-18T11:46:34.617 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:34.617 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:34.617 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-02-18 11:46:24.436353827 +0000 2024-02-18T11:46:34.618 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-02-18 11:44:32.273792235 +0000 2024-02-18T11:46:34.618 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-02-18 11:44:32.273792235 +0000 2024-02-18T11:46:34.618 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-02-18 11:44:32.273792235 +0000 2024-02-18T11:46:34.618 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T11:46:34.682 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-02-18T11:46:34.682 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-02-18T11:46:34.682 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000220734 s, 2.3 MB/s 2024-02-18T11:46:34.683 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T11:46:34.739 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:34.739 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:34.739 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T11:46:34.754 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T11:46:34.755 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-02-18T11:46:34.810 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T11:46:34.810 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:34.810 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-18T11:46:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-02-18 11:46:21.856005424 +0000 2024-02-18T11:46:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-02-18 11:44:31.237699956 +0000 2024-02-18T11:46:34.811 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-02-18 11:44:31.237699956 +0000 2024-02-18T11:46:34.811 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-02-18 11:44:31.237699956 +0000 2024-02-18T11:46:34.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T11:46:34.876 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-02-18T11:46:34.876 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-02-18T11:46:34.876 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000258026 s, 2.0 MB/s 2024-02-18T11:46:34.878 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T11:46:34.933 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-02-18T11:46:34.988 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-02-18 11:46:21.856005424 +0000 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-02-18 11:44:31.584690320 +0000 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-02-18 11:44:31.584690320 +0000 2024-02-18T11:46:34.989 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-02-18 11:44:31.584690320 +0000 2024-02-18T11:46:34.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T11:46:35.054 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-02-18T11:46:35.054 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-02-18T11:46:35.054 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000190371 s, 2.7 MB/s 2024-02-18T11:46:35.055 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T11:46:35.111 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-02-18T11:46:35.166 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T11:46:35.166 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:35.166 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-18T11:46:35.166 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:35.166 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:35.166 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-02-18 11:46:21.857005395 +0000 2024-02-18T11:46:35.167 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-02-18 11:44:31.948680212 +0000 2024-02-18T11:46:35.167 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-02-18 11:44:31.948680212 +0000 2024-02-18T11:46:35.167 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-02-18 11:44:31.948680212 +0000 2024-02-18T11:46:35.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T11:46:35.232 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-02-18T11:46:35.232 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-02-18T11:46:35.232 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000208442 s, 2.5 MB/s 2024-02-18T11:46:35.233 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T11:46:35.288 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-02-18T11:46:35.343 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T11:46:35.343 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T11:46:35.343 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-18T11:46:35.343 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T11:46:35.344 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-02-18T11:46:35.344 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-02-18 11:46:21.857005395 +0000 2024-02-18T11:46:35.344 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-02-18 11:44:32.292670660 +0000 2024-02-18T11:46:35.344 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-02-18 11:44:32.292670660 +0000 2024-02-18T11:46:35.344 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-02-18 11:44:32.292670660 +0000 2024-02-18T11:46:35.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T11:46:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-02-18T11:46:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-02-18T11:46:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.00020612 s, 2.5 MB/s 2024-02-18T11:46:35.411 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T11:46:35.467 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:35.467 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-18T11:46:35.467 INFO:tasks.ceph:Generating config... 2024-02-18T11:46:35.468 INFO:tasks.ceph:[client] client mount timeout = 600 2024-02-18T11:46:35.469 INFO:tasks.ceph:[client] debug client = 20 2024-02-18T11:46:35.469 INFO:tasks.ceph:[client] debug ms = 1 2024-02-18T11:46:35.469 INFO:tasks.ceph:[client] pid file = 2024-02-18T11:46:35.469 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-02-18T11:46:35.469 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-02-18T11:46:35.469 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-02-18T11:46:35.469 INFO:tasks.ceph:[mds] debug mds = 20 2024-02-18T11:46:35.469 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-02-18T11:46:35.469 INFO:tasks.ceph:[mds] debug ms = 1 2024-02-18T11:46:35.469 INFO:tasks.ceph:[mds] mds debug frag = True 2024-02-18T11:46:35.469 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mgr] debug ms = 1 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mon] debug mon = 20 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mon] debug ms = 1 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mon] debug paxos = 20 2024-02-18T11:46:35.470 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-02-18T11:46:35.470 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-02-18T11:46:35.470 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] debug ms = 1 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] debug osd = 20 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-02-18T11:46:35.471 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-02-18T11:46:35.472 INFO:tasks.ceph:Setting up mon.a... 2024-02-18T11:46:35.472 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-02-18T11:46:35.523 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.keyring 2024-02-18T11:46:35.526 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-02-18T11:46:35.570 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-18T11:46:35.597 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.116'), ('mon.b', '172.21.15.143'), ('mon.c', '[v2:172.21.15.143:3301,v1:172.21.15.143:6790]')] 2024-02-18T11:46:35.598 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.116,172.21.15.143,[v2:172.21.15.143:3301,v1:172.21.15.143:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-02-18T11:46:35.599 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:35.599 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-02-18T11:46:35.652 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.116 --add b 172.21.15.143 --addv c '[v2:172.21.15.143:3301,v1:172.21.15.143:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:35.736 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:35.736 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: generated fsid 82e3e917-bb19-48fa-a66a-62df77a0f413 2024-02-18T11:46:35.736 INFO:teuthology.orchestra.run.smithi116.stdout:setting min_mon_release = quincy 2024-02-18T11:46:35.736 INFO:teuthology.orchestra.run.smithi116.stdout:epoch 0 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:fsid 82e3e917-bb19-48fa-a66a-62df77a0f413 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:last_changed 2024-02-18T11:46:35.744646+0000 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:created 2024-02-18T11:46:35.744646+0000 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:min_mon_release 17 (quincy) 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:election_strategy: 1 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.b 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:2: [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] mon.c 2024-02-18T11:46:35.737 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-02-18T11:46:35.738 DEBUG:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-02-18T11:46:35.752 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 82e3e917-bb19-48fa-a66a-62df77a0f413... 2024-02-18T11:46:35.753 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-18T11:46:35.795 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-02-18T11:46:35.843 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-02-18T11:46:35.843 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-02-18T11:46:35.843 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-18T11:46:35.843 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-02-18T11:46:35.843 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.843 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.844 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-18T11:46:35.845 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.143,[v2:172.21.15.143:3301,v1:172.21.15.143:6790]" 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 82e3e917-bb19-48fa-a66a-62df77a0f413 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.846 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.847 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-02-18T11:46:35.848 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-02-18T11:46:35.849 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-02-18T11:46:35.850 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.851 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-02-18T11:46:35.852 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-02-18T11:46:35.853 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-02-18T11:46:35.854 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2024-02-18T11:46:35.855 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2024-02-18T11:46:35.856 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-02-18T11:46:35.857 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.116,172.21.15.143,[v2:172.21.15.143:3301,v1:172.21.15.143:6790]" 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on skipped message = False 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 82e3e917-bb19-48fa-a66a-62df77a0f413 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.858 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2024-02-18T11:46:35.859 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2024-02-18T11:46:35.860 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 20 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 20 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 10 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2024-02-18T11:46:35.861 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-02-18T11:46:35.862 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2024-02-18T11:46:35.863 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2024-02-18T11:46:35.864 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2024-02-18T11:46:35.865 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2024-02-18T11:46:35.866 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2024-02-18T11:46:35.867 INFO:tasks.ceph:Creating admin key on mon.a... 2024-02-18T11:46:35.868 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-02-18T11:46:35.964 INFO:tasks.ceph:Copying monmap to all nodes... 2024-02-18T11:46:35.964 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:35.964 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-02-18T11:46:35.981 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:35.981 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-02-18T11:46:36.035 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:46:36.036 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:36.036 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-18T11:46:36.036 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-18T11:46:36.115 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:46:36.115 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:36.170 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:46:36.170 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:36.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2024-02-18T11:46:36.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-02-18T11:46:36.209 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:46:36.209 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:36.264 INFO:tasks.ceph:Setting up mon nodes... 2024-02-18T11:46:36.265 INFO:tasks.ceph:Setting up mgr nodes... 2024-02-18T11:46:36.265 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-02-18T11:46:36.319 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-02-18T11:46:36.322 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-02-18T11:46:36.393 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-02-18T11:46:36.395 INFO:tasks.ceph:Setting up mds nodes... 2024-02-18T11:46:36.395 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-02-18T11:46:36.449 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-02-18T11:46:36.452 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-02-18T11:46:36.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-02-18T11:46:36.573 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-02-18T11:46:36.576 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-02-18T11:46:36.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-02-18T11:46:36.657 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-02-18T11:46:36.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-02-18T11:46:36.686 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-02-18T11:46:36.781 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-02-18T11:46:36.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-02-18T11:46:36.809 INFO:tasks.ceph_client:Setting up client nodes... 2024-02-18T11:46:36.810 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-18T11:46:36.852 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-02-18T11:46:36.867 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-18T11:46:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-02-18T11:46:36.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-18T11:46:37.006 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-02-18T11:46:37.021 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-02-18T11:46:37.103 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-02-18T11:46:37.118 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-02-18T11:46:37.118 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-02-18T11:46:37.119 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-02-18T11:46:37.145 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:37.146 INFO:tasks.ceph:role: osd.0 2024-02-18T11:46:37.146 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:46:37.146 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-18T11:46:37.219 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:37.220 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:37.709 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-02-18T11:46:37.711 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-02-18T11:46:37.711 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-02-18T11:46:37.826 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-02-18T11:46:37.859 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-02-18T11:46:37.927 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:37.928 INFO:tasks.ceph:role: osd.1 2024-02-18T11:46:37.928 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:46:37.928 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:38.002 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:38.003 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:38.475 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-02-18T11:46:38.477 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-02-18T11:46:38.477 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-02-18T11:46:38.570 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-02-18T11:46:38.601 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-02-18T11:46:38.670 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:38.670 INFO:tasks.ceph:role: osd.2 2024-02-18T11:46:38.670 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:46:38.670 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:38.744 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:39.230 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-02-18T11:46:39.232 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-02-18T11:46:39.233 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-02-18T11:46:39.367 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-02-18T11:46:39.398 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-02-18T11:46:39.465 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:39.465 INFO:tasks.ceph:role: osd.3 2024-02-18T11:46:39.465 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:46:39.465 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:39.539 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:39.540 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:39.540 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:40.021 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-02-18T11:46:40.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-02-18T11:46:40.023 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-02-18T11:46:40.139 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-02-18T11:46:40.173 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:40.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:40.274+0000 7f1eb6338640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-02-18T11:46:40.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:40.274+0000 7f1eb6338640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-02-18T11:46:40.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:40.274+0000 7f1eb6338640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-18T11:46:40.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:40.274+0000 7f1eb6338640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-18T11:46:40.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:40.274+0000 7f1eb6338640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-02-18T11:46:40.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:40.275+0000 7f1eb6338640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-02-18T11:46:43.114 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-18T11:46:43.147 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:43.246+0000 7ff92b024640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-02-18T11:46:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:43.246+0000 7ff92b024640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-02-18T11:46:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:43.246+0000 7ff92b024640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-18T11:46:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:43.246+0000 7ff92b024640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-18T11:46:43.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:43.246+0000 7ff92b024640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-02-18T11:46:43.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:43.246+0000 7ff92b024640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-02-18T11:46:46.084 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-18T11:46:46.115 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:46.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:46.213+0000 7f3f5a6a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-02-18T11:46:46.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:46.213+0000 7f3f5a6a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-02-18T11:46:46.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:46.213+0000 7f3f5a6a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-18T11:46:46.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:46.213+0000 7f3f5a6a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-18T11:46:46.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:46.213+0000 7f3f5a6a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-02-18T11:46:46.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:46.214+0000 7f3f5a6a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-02-18T11:46:49.050 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-18T11:46:49.078 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:49.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:49.171+0000 7facf6015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-02-18T11:46:49.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:49.172+0000 7facf6015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-02-18T11:46:49.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:49.172+0000 7facf6015640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-18T11:46:49.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:49.172+0000 7facf6015640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-18T11:46:49.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:49.172+0000 7facf6015640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-02-18T11:46:49.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:46:49.172+0000 7facf6015640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-02-18T11:46:52.009 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-18T11:46:52.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-02-18T11:46:52.073 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:52.073 INFO:tasks.ceph:role: osd.4 2024-02-18T11:46:52.073 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:46:52.073 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:52.151 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:52.152 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:52.152 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:52.152 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:52.652 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-02-18T11:46:52.654 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-02-18T11:46:52.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-02-18T11:46:52.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-02-18T11:46:52.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-02-18T11:46:52.870 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:52.870 INFO:tasks.ceph:role: osd.5 2024-02-18T11:46:52.870 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:46:52.871 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-02-18T11:46:52.966 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:52.966 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:52.966 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:52.966 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:52.966 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:52.966 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:52.967 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:52.967 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:52.967 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:52.967 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:53.408 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-02-18T11:46:53.409 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-02-18T11:46:53.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-02-18T11:46:53.462 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-02-18T11:46:53.492 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-02-18T11:46:53.559 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:53.559 INFO:tasks.ceph:role: osd.6 2024-02-18T11:46:53.559 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:46:53.559 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:53.634 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:54.131 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-02-18T11:46:54.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-02-18T11:46:54.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-02-18T11:46:54.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-02-18T11:46:54.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-02-18T11:46:54.376 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-02-18T11:46:54.377 INFO:tasks.ceph:role: osd.7 2024-02-18T11:46:54.377 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:46:54.377 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-02-18T11:46:54.451 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-02-18T11:46:54.452 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-02-18T11:46:54.979 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-02-18T11:46:54.981 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-02-18T11:46:54.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-02-18T11:46:55.139 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-02-18T11:46:55.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:55.288+0000 7fa50e813640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-02-18T11:46:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:55.289+0000 7fa50e813640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-02-18T11:46:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:55.289+0000 7fa50e813640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-18T11:46:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:55.289+0000 7fa50e813640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-18T11:46:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:55.289+0000 7fa50e813640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-02-18T11:46:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:55.289+0000 7fa50e813640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-02-18T11:46:58.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-18T11:46:58.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:46:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:58.281+0000 7efdf5716640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-02-18T11:46:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:58.281+0000 7efdf5716640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-02-18T11:46:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:58.281+0000 7efdf5716640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-18T11:46:58.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:58.281+0000 7efdf5716640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-18T11:46:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:58.281+0000 7efdf5716640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-02-18T11:46:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:46:58.282+0000 7efdf5716640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-02-18T11:47:01.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-18T11:47:01.159 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:47:01.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:01.256+0000 7f11fb00b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-02-18T11:47:01.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:01.256+0000 7f11fb00b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-02-18T11:47:01.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:01.256+0000 7f11fb00b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-18T11:47:01.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:01.256+0000 7f11fb00b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-18T11:47:01.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:01.256+0000 7f11fb00b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-02-18T11:47:01.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:01.256+0000 7f11fb00b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-02-18T11:47:03.893 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:47:03.896 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:47:04.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-02-18T11:47:04.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:47:04.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:04.226+0000 7f3c6768b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-02-18T11:47:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:04.227+0000 7f3c6768b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-02-18T11:47:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:04.227+0000 7f3c6768b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-18T11:47:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:04.227+0000 7f3c6768b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-18T11:47:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:04.227+0000 7f3c6768b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-02-18T11:47:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:04.227+0000 7f3c6768b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-02-18T11:47:07.071 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-02-18T11:47:07.102 INFO:tasks.ceph:Reading keys from all nodes... 2024-02-18T11:47:07.103 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.103 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-02-18T11:47:07.134 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.134 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-02-18T11:47:07.203 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.204 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-02-18T11:47:07.273 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.273 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-02-18T11:47:07.342 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.342 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-02-18T11:47:07.411 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.411 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-02-18T11:47:07.478 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-02-18T11:47:07.545 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-02-18T11:47:07.577 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-02-18T11:47:07.647 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-02-18T11:47:07.717 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-02-18T11:47:07.786 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-02-18T11:47:07.854 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.854 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-02-18T11:47:07.923 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:07.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-02-18T11:47:07.989 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:07.989 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-02-18T11:47:08.005 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:08.005 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-02-18T11:47:08.062 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:08.062 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-02-18T11:47:08.076 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:08.077 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-02-18T11:47:08.131 INFO:tasks.ceph:Adding keys to all mons... 2024-02-18T11:47:08.132 DEBUG:teuthology.orchestra.run.smithi116:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-18T11:47:08.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2024-02-18T11:47:08.174 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlo3OFExAA02E+7DFvUakguPorwbhBUA== 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlyhFMGxAAMeF+iwTSDh0X1A3ZXxuDTg== 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlNJexIhAAp4vwOjUHQEcuqo2D72RV+Q== 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAg7tFlXmZiEBAAfS+Eercp/1xgAcPRd14DoQ== 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAj7tFl+savDhAAicZUH6PgxZwa1O26SjOKhg== 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2024-02-18T11:47:08.175 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAm7tFlYhrKDBAALKMstc0EPp/QQlFRwhxBwA== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAp7tFlmZ1TChAAVLXNxeffx0iciG6tBX+7JA== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.x] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlXYfnFxAAoKC7TGSsy6LJIlhW35+8rg== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlN/msJxAAJqlgK7WxiuEWdZrEG8v2RQ== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlBHwKLxAADXZv7IdT3E6Hhuv1dtwRZg== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.4] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAv7tFlCltEERAA1CFMxClfgFn3GuPqDjkIWA== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.5] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAy7tFlzGTgEBAADx0h93HsLgqsh9nT3XThVQ== 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.6] 2024-02-18T11:47:08.176 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQA17tFllAVeDxAAOduu0ARW9LFB4aCb5f59Pg== 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.7] 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQA47tFly22eDRAAlSlKL15kQi7t8ftXRZY6Yw== 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlVhFMMxAAcC+CBcaJjyYSC7rVBwyvOw== 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout:[client.2] 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlyCgUORAA3NQgD2krY5qBoJd79bNekg== 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout:[client.1] 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAd7tFloBHcABAAXBVf3C+t7Ia28G5cwLYxGg== 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout:[client.3] 2024-02-18T11:47:08.177 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAd7tFleeykBhAAS4gRO+FydukZBSNIOL8gOA== 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.y] 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlo3OFExAA02E+7DFvUakguPorwbhBUA== 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlyhFMGxAAMeF+iwTSDh0X1A3ZXxuDTg== 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlNJexIhAAp4vwOjUHQEcuqo2D72RV+Q== 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAg7tFlXmZiEBAAfS+Eercp/1xgAcPRd14DoQ== 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2024-02-18T11:47:08.197 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAj7tFl+savDhAAicZUH6PgxZwa1O26SjOKhg== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAm7tFlYhrKDBAALKMstc0EPp/QQlFRwhxBwA== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.3] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAp7tFlmZ1TChAAVLXNxeffx0iciG6tBX+7JA== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlXYfnFxAAoKC7TGSsy6LJIlhW35+8rg== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlN/msJxAAJqlgK7WxiuEWdZrEG8v2RQ== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlBHwKLxAADXZv7IdT3E6Hhuv1dtwRZg== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.4] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAv7tFlCltEERAA1CFMxClfgFn3GuPqDjkIWA== 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.5] 2024-02-18T11:47:08.198 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAy7tFlzGTgEBAADx0h93HsLgqsh9nT3XThVQ== 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.6] 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQA17tFllAVeDxAAOduu0ARW9LFB4aCb5f59Pg== 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.7] 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQA47tFly22eDRAAlSlKL15kQi7t8ftXRZY6Yw== 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlVhFMMxAAcC+CBcaJjyYSC7rVBwyvOw== 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:[client.2] 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAc7tFlyCgUORAA3NQgD2krY5qBoJd79bNekg== 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:[client.1] 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAd7tFloBHcABAAXBVf3C+t7Ia28G5cwLYxGg== 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout:[client.3] 2024-02-18T11:47:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQAd7tFleeykBhAAS4gRO+FydukZBSNIOL8gOA== 2024-02-18T11:47:08.200 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-18T11:47:08.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-18T11:47:08.292 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.349 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.405 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.407 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.458 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.513 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.567 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.569 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.621 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-18T11:47:08.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-02-18T11:47:08.675 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.677 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.732 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-02-18T11:47:08.788 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.842 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.844 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.896 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.950 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:08.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-02-18T11:47:09.004 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.061 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.121 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.124 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.179 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.181 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-02-18T11:47:09.236 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-02-18T11:47:09.236 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-02-18T11:47:09.262 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-18T11:47:09.358 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-02-18T11:47:09.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-02-18T11:47:09.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-18T11:47:09.506 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-02-18T11:47:09.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-02-18T11:47:09.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-02-18T11:47:09.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-02-18T11:47:09.723 DEBUG:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:47:09.725 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-02-18T11:47:09.778 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-02-18T11:47:09.778 INFO:tasks.ceph.mon.a:Restarting daemon 2024-02-18T11:47:09.778 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-02-18T11:47:09.780 INFO:tasks.ceph.mon.a:Started 2024-02-18T11:47:09.780 INFO:tasks.ceph.mon.b:Restarting daemon 2024-02-18T11:47:09.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-02-18T11:47:09.821 INFO:tasks.ceph.mon.b:Started 2024-02-18T11:47:09.821 INFO:tasks.ceph.mon.c:Restarting daemon 2024-02-18T11:47:09.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-02-18T11:47:09.823 INFO:tasks.ceph.mon.c:Started 2024-02-18T11:47:09.823 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-02-18T11:47:09.824 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-02-18T11:47:09.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-02-18T11:47:09.826 INFO:tasks.ceph.mgr.y:Started 2024-02-18T11:47:09.826 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-02-18T11:47:09.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-02-18T11:47:09.828 INFO:tasks.ceph.mgr.x:Started 2024-02-18T11:47:09.828 INFO:tasks.ceph:Setting crush tunables to default 2024-02-18T11:47:09.828 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd crush tunables default 2024-02-18T11:47:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.936+0000 7f181cb4b640 1 Processor -- start 2024-02-18T11:47:09.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.936+0000 7f181cb4b640 1 -- start start 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.937+0000 7f181cb4b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 0x7f18180704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.937+0000 7f181cb4b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 0x7f1818167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.937+0000 7f181cb4b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180cd540 con 0x7f1818072d60 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.937+0000 7f181cb4b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18180b3de0 con 0x7f181815da30 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.937+0000 7f181cb4b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18180731a0 con 0x7f181806b270 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.937+0000 7f181659c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 0x7f18180704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f181659c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 0x7f18180704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33438/0 (socket says 172.21.15.116:33438) 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f1815d9b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1818072d60 0x7f181806ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55100/0 (socket says 172.21.15.116:55100) 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f181659c640 1 -- 172.21.15.116:0/2230646749 learned_addr learned my addr 172.21.15.116:0/2230646749 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f181659c640 1 -- 172.21.15.116:0/2230646749 >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 msgr2=0x7f18180704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-02-18T11:47:09.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f181659c640 1 -- 172.21.15.116:0/2230646749 >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 msgr2=0x7f18180704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f181659c640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 0x7f18180704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f1816d9d640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 0x7f1818167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f181659c640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 0x7f18180704a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f1816d9d640 1 -- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 msgr2=0x7f1818167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f1816d9d640 1 -- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 msgr2=0x7f1818167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f1816d9d640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 0x7f1818167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-18T11:47:09.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:09.938+0000 7f1816d9d640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 0x7f1818167550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-18T11:47:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351879512 0 0) 0x7f18180cd540 con 0x7f1818072d60 2024-02-18T11:47:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1800003520 con 0x7f1818072d60 2024-02-18T11:47:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 73096548 0 0) 0x7f1800003520 con 0x7f1818072d60 2024-02-18T11:47:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 msgr2=0x7f1818167550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.143:3300/0 conn(0x7f181806b270 0x7f1818167550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 msgr2=0x7f18180704a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.115+0000 7f181559a640 1 --2- 172.21.15.116:0/2230646749 >> v2:172.21.15.116:3300/0 conn(0x7f181815da30 0x7f18180704a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181815de10 con 0x7f1818072d60 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f180c0019f0 con 0x7f1818072d60 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 >> v1:172.21.15.116:6789/0 conn(0x7f1818072d60 legacy=0x7f181806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1800003970 0x7f1800003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18000047c0 0x7f1800004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800005180 0x7f180000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b3de0 con 0x7f18000047c0 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180731a0 con 0x7f1800003970 2024-02-18T11:47:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180000c080 con 0x7f1800005180 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181659c640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1800003970 0x7f1800003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f1816d9d640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800005180 0x7f180000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f1815d9b640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18000047c0 0x7f1800004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181659c640 1 -- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800005180 msgr2=0x7f180000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181659c640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800005180 0x7f180000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181659c640 1 -- 172.21.15.116:0/2230646749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18000047c0 msgr2=0x7f1800004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181659c640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18000047c0 0x7f1800004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f181659c640 1 -- 172.21.15.116:0/2230646749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1800003520 con 0x7f1800003970 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.116+0000 7f1816d9d640 1 -- 172.21.15.116:0/2230646749 reap_dead start 2024-02-18T11:47:10.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.117+0000 7f181659c640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1800003970 0x7f1800003d90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1810009890 tx=0x7f18000057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e564d1bce67bd36c server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:10.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.119+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810030680 con 0x7f1800003970 2024-02-18T11:47:10.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.119+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100307e0 con 0x7f1800003970 2024-02-18T11:47:10.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.119+0000 7f181559a640 1 -- 172.21.15.116:0/2230646749 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810030b90 con 0x7f1800003970 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 -- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1800003970 msgr2=0x7f1800003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1800003970 0x7f1800003d90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1810009890 tx=0x7f18000057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 -- 172.21.15.116:0/2230646749 shutdown_connections 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 --2- 172.21.15.116:0/2230646749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1800003970 0x7f1800003d90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 -- 172.21.15.116:0/2230646749 >> 172.21.15.116:0/2230646749 conn(0x7f18180b23f0 msgr2=0x7f18180b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 -- 172.21.15.116:0/2230646749 shutdown_connections 2024-02-18T11:47:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.120+0000 7f181cb4b640 1 -- 172.21.15.116:0/2230646749 wait complete. 2024-02-18T11:47:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 Processor -- start 2024-02-18T11:47:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 -- start start 2024-02-18T11:47:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800003970 0x7f18181524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 0x7f18180b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f181813f790 0x7f181813fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18181403e0 con 0x7f18180cc2f0 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1818140150 con 0x7f181813f790 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181cb4b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1818152bb0 con 0x7f1800003970 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f1815d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 0x7f18180b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f1815d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 0x7f18180b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33488/0 (socket says 172.21.15.116:33488) 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f1815d9b640 1 -- 172.21.15.116:0/2463955267 learned_addr learned my addr 172.21.15.116:0/2463955267 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f181659c640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800003970 0x7f18181524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.122+0000 7f1816d9d640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f181813f790 0x7f181813fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1815d9b640 1 -- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800003970 msgr2=0x7f18181524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1815d9b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800003970 0x7f18181524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1815d9b640 1 -- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f181813f790 msgr2=0x7f181813fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1815d9b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f181813f790 0x7f181813fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1815d9b640 1 -- 172.21.15.116:0/2463955267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1818152e90 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1816d9d640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f181813f790 0x7f181813fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1815d9b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 0x7f18180b1430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f180c0076c0 tx=0x7f180c007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00fba0 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c00fd20 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1818153170 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c013630 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18180cc770 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f180c013790 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.123+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f180c01db10 con 0x7f18180cc2f0 2024-02-18T11:47:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.124+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18180cc900 con 0x7f18180cc2f0 2024-02-18T11:47:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.126+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f18180cc900 con 0x7f18180cc2f0 2024-02-18T11:47:10.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.191+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1818140550 con 0x7f18180cc2f0 2024-02-18T11:47:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.193+0000 7f1806ffd640 1 -- 172.21.15.116:0/2463955267 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1818140550 con 0x7f18180cc2f0 2024-02-18T11:47:10.186 INFO:teuthology.orchestra.run.smithi116.stderr:adjusted tunables profile to default 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 msgr2=0x7f18180b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 0x7f18180b1430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f180c0076c0 tx=0x7f180c007b80 comp rx=0 tx=0).stop 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 shutdown_connections 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f181813f790 0x7f181813fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18180cc2f0 0x7f18180b1430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 --2- 172.21.15.116:0/2463955267 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1800003970 0x7f18181524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 >> 172.21.15.116:0/2463955267 conn(0x7f18180b23f0 msgr2=0x7f18180af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 shutdown_connections 2024-02-18T11:47:10.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:10.194+0000 7f181cb4b640 1 -- 172.21.15.116:0/2463955267 wait complete. 2024-02-18T11:47:10.198 INFO:tasks.ceph:check_enable_crimson: False 2024-02-18T11:47:10.198 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-02-18T11:47:10.199 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:10.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-02-18T11:47:10.226 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:10.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-02-18T11:47:10.293 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:10.293 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-02-18T11:47:10.360 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:10.360 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-02-18T11:47:10.427 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:10.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-02-18T11:47:10.455 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:10.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-02-18T11:47:10.522 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:10.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-02-18T11:47:10.583 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:10.590+0000 7ff775241040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:47:10.588 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:47:10.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-02-18T11:47:10.635 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:10.642+0000 7ff775241040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:47:10.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new 7af0914b-25ef-478f-812e-7e545d4216b4 0 2024-02-18T11:47:10.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 Processor -- start 2024-02-18T11:47:10.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 -- start start 2024-02-18T11:47:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f01617e0 0x7fc3f00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 0x7fc3f00d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f00d1ee0 con 0x7fc3f015dbf0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f00cf110 con 0x7fc3f00cec80 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.804+0000 7fc3f53c7640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f0161dc0 con 0x7fc3f01617e0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3ef7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 0x7fc3f00d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3e67fc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f01617e0 0x7fc3f00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3ef7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 0x7fc3f00d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47922/0 (socket says 172.21.15.143:47922) 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3ef7fe640 1 -- 172.21.15.143:0/2005504543 learned_addr learned my addr 172.21.15.143:0/2005504543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3e67fc640 1 -- 172.21.15.143:0/2005504543 >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 msgr2=0x7fc3f00d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3e67fc640 1 --2- 172.21.15.143:0/2005504543 >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 0x7fc3f00d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3e67fc640 1 -- 172.21.15.143:0/2005504543 >> v1:172.21.15.116:6789/0 conn(0x7fc3f015dbf0 legacy=0x7fc3f00b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3e67fc640 1 -- 172.21.15.143:0/2005504543 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f01620a0 con 0x7fc3f01617e0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.805+0000 7fc3ef7fe640 1 --2- 172.21.15.143:0/2005504543 >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 0x7fc3f00d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3e67fc640 1 --2- 172.21.15.143:0/2005504543 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f01617e0 0x7fc3f00ce740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80097a0 tx=0x7fc3d802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4c78dc83f68537 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3ee7fc640 1 -- 172.21.15.143:0/2005504543 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d802e070 con 0x7fc3f01617e0 2024-02-18T11:47:10.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3ee7fc640 1 -- 172.21.15.143:0/2005504543 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d80315a0 con 0x7fc3f01617e0 2024-02-18T11:47:10.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3ee7fc640 1 -- 172.21.15.143:0/2005504543 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8031930 con 0x7fc3f01617e0 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2005504543 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f01617e0 msgr2=0x7fc3f00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2005504543 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f01617e0 0x7fc3f00ce740 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80097a0 tx=0x7fc3d802b840 comp rx=0 tx=0).stop 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2005504543 shutdown_connections 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2005504543 >> v2:172.21.15.143:3300/0 conn(0x7fc3f00cec80 0x7fc3f00d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2005504543 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f01617e0 0x7fc3f00ce740 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.806+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2005504543 >> 172.21.15.143:0/2005504543 conn(0x7fc3f00b5480 msgr2=0x7fc3f00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.807+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2005504543 shutdown_connections 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.807+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2005504543 wait complete. 2024-02-18T11:47:10.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.808+0000 7fc3f53c7640 1 Processor -- start 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.808+0000 7fc3f53c7640 1 -- start start 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.808+0000 7fc3f53c7640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 0x7fc3f00aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3f015dbf0 0x7fc3f00c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 0x7fc3f0159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015a030 con 0x7fc3f015dbf0 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015a260 con 0x7fc3f01617e0 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015a540 con 0x7fc3f00cec80 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3ef7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 0x7fc3f0159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 0x7fc3f00aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3ef7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 0x7fc3f0159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47934/0 (socket says 172.21.15.143:47934) 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 0x7fc3f00aa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39170/0 (socket says 172.21.15.143:39170) 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3ef7fe640 1 -- 172.21.15.143:0/2239907132 learned_addr learned my addr 172.21.15.143:0/2239907132 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:10.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3e67fc640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3f015dbf0 0x7fc3f00c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 -- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 msgr2=0x7fc3f0159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 0x7fc3f0159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 -- 172.21.15.143:0/2239907132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3f015dbf0 msgr2=0x7fc3f00c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3f015dbf0 0x7fc3f00c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 -- 172.21.15.143:0/2239907132 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8009b30 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3ef7fe640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 0x7fc3f0159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3eeffd640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 0x7fc3f00aa6e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c340 tx=0x7fc3e000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00052d0 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e0005450 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d8009400 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0007260 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.809+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f015e150 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.810+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc3e00073c0 con 0x7fc3f00cec80 2024-02-18T11:47:10.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.810+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc3e00039b0 con 0x7fc3f00cec80 2024-02-18T11:47:10.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.810+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f0067840 con 0x7fc3f00cec80 2024-02-18T11:47:10.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.814+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc3e0020060 con 0x7fc3f00cec80 2024-02-18T11:47:10.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.886+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7af0914b-25ef-478f-812e-7e545d4216b4", "id": 0} v 0) v1 -- 0x7fc3f0074520 con 0x7fc3f00cec80 2024-02-18T11:47:10.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.890+0000 7fc3e7fff640 1 -- 172.21.15.143:0/2239907132 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7af0914b-25ef-478f-812e-7e545d4216b4", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc3e0003b70 con 0x7fc3f00cec80 2024-02-18T11:47:10.883 INFO:teuthology.orchestra.run.smithi143.stdout:0 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 msgr2=0x7fc3f00aa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 0x7fc3f00aa6e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c340 tx=0x7fc3e000c740 comp rx=0 tx=0).stop 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 shutdown_connections 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3f01617e0 0x7fc3f0159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3f015dbf0 0x7fc3f00c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 --2- 172.21.15.143:0/2239907132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3f00cec80 0x7fc3f00aa6e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 >> 172.21.15.143:0/2239907132 conn(0x7fc3f00b5480 msgr2=0x7fc3f00b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:10.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 shutdown_connections 2024-02-18T11:47:10.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:10.891+0000 7fc3f53c7640 1 -- 172.21.15.143:0/2239907132 wait complete. 2024-02-18T11:47:10.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new f78069a3-e4be-4438-9956-85482d8671c1 1 2024-02-18T11:47:10.971 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:10.978+0000 7f88086c3040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-18T11:47:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.002+0000 7f833e0df640 1 Processor -- start 2024-02-18T11:47:10.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.002+0000 7f833e0df640 1 -- start start 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f833e0df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f83380ae880 0x7f83380aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f833e0df640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83380cfc10 con 0x7f833815c380 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f833e0df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83380668f0 con 0x7f83380cdca0 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f833e0df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833815ce70 con 0x7f83380ae880 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f832ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f83380ae880 0x7f83380aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f832ffff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f83380ae880 0x7f83380aecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47948/0 (socket says 172.21.15.143:47948) 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f832ffff640 1 -- 172.21.15.143:0/3938684865 learned_addr learned my addr 172.21.15.143:0/3938684865 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f832ffff640 1 -- 172.21.15.143:0/3938684865 >> v1:172.21.15.143:6789/0 conn(0x7f83380cdca0 legacy=0x7f83380ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f832ffff640 1 -- 172.21.15.143:0/3938684865 >> v1:172.21.15.116:6789/0 conn(0x7f833815c380 legacy=0x7f833815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.003+0000 7f832ffff640 1 -- 172.21.15.143:0/3938684865 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380cf930 con 0x7f83380ae880 2024-02-18T11:47:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.004+0000 7f832ffff640 1 --2- 172.21.15.143:0/3938684865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83380ae880 0x7f83380aecc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8320009870 tx=0x7f832002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316681a03bf74fde server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.004+0000 7f8336ffd640 1 -- 172.21.15.143:0/3938684865 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832002f840 con 0x7f83380ae880 2024-02-18T11:47:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.004+0000 7f8336ffd640 1 -- 172.21.15.143:0/3938684865 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832002f9a0 con 0x7f83380ae880 2024-02-18T11:47:10.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.004+0000 7f8336ffd640 1 -- 172.21.15.143:0/3938684865 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832002fd30 con 0x7f83380ae880 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 -- 172.21.15.143:0/3938684865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83380ae880 msgr2=0x7f83380aecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 --2- 172.21.15.143:0/3938684865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83380ae880 0x7f83380aecc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8320009870 tx=0x7f832002ba20 comp rx=0 tx=0).stop 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 -- 172.21.15.143:0/3938684865 shutdown_connections 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 --2- 172.21.15.143:0/3938684865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83380ae880 0x7f83380aecc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 -- 172.21.15.143:0/3938684865 >> 172.21.15.143:0/3938684865 conn(0x7f83380cc820 msgr2=0x7f83380ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 -- 172.21.15.143:0/3938684865 shutdown_connections 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.005+0000 7f833e0df640 1 -- 172.21.15.143:0/3938684865 wait complete. 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.006+0000 7f833e0df640 1 Processor -- start 2024-02-18T11:47:10.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.006+0000 7f833e0df640 1 -- start start 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f833e0df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83380ae880 0x7f83380cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f833e0df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83380cdca0 0x7f833814f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f833e0df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 0x7f83380b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f833e0df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338154af0 con 0x7f83380ae880 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f833e0df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8338153d40 con 0x7f833815c380 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f833e0df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83380cbb50 con 0x7f83380cdca0 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 0x7f83380b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f832ffff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83380cdca0 0x7f833814f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 0x7f83380b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47950/0 (socket says 172.21.15.143:47950) 2024-02-18T11:47:11.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 -- 172.21.15.143:0/758793581 learned_addr learned my addr 172.21.15.143:0/758793581 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f83377fe640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83380ae880 0x7f83380cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 -- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83380cdca0 msgr2=0x7f833814f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83380cdca0 0x7f833814f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 -- 172.21.15.143:0/758793581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83380ae880 msgr2=0x7f83380cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83380ae880 0x7f83380cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f8337fff640 1 -- 172.21.15.143:0/758793581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83200094d0 con 0x7f833815c380 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f83377fe640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83380ae880 0x7f83380cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.007+0000 7f832ffff640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83380cdca0 0x7f833814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f8337fff640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 0x7f83380b09a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f833000c7d0 tx=0x7f833000cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330003b70 con 0x7f833815c380 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8330003cf0 con 0x7f833815c380 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833000aa90 con 0x7f833815c380 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83380b0510 con 0x7f833815c380 2024-02-18T11:47:11.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8338154810 con 0x7f833815c380 2024-02-18T11:47:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f833000abf0 con 0x7f833815c380 2024-02-18T11:47:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.008+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8330012550 con 0x7f833815c380 2024-02-18T11:47:11.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.009+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83380b02e0 con 0x7f833815c380 2024-02-18T11:47:11.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.013+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f83380b02e0 con 0x7f833815c380 2024-02-18T11:47:11.029 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.037+0000 7f88086c3040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T11:47:11.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.085+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f78069a3-e4be-4438-9956-85482d8671c1", "id": 1} v 0) v1 -- 0x7f83380b06a0 con 0x7f833815c380 2024-02-18T11:47:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.090+0000 7f8334ff9640 1 -- 172.21.15.143:0/758793581 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f78069a3-e4be-4438-9956-85482d8671c1", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f83300050d0 con 0x7f833815c380 2024-02-18T11:47:11.083 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-02-18T11:47:11.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 msgr2=0x7f83380b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 0x7f83380b09a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f833000c7d0 tx=0x7f833000cc90 comp rx=0 tx=0).stop 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 shutdown_connections 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f833815c380 0x7f83380b09a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83380cdca0 0x7f833814f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 --2- 172.21.15.143:0/758793581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83380ae880 0x7f83380cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 >> 172.21.15.143:0/758793581 conn(0x7f83380cc820 msgr2=0x7f833815c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 shutdown_connections 2024-02-18T11:47:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.091+0000 7f833e0df640 1 -- 172.21.15.143:0/758793581 wait complete. 2024-02-18T11:47:11.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new 391fd049-5adf-469c-a031-e1d2cf36f723 2 2024-02-18T11:47:11.164 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.171+0000 7ff775241040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:47:11.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 Processor -- start 2024-02-18T11:47:11.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 -- start start 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d4406b270 0x7f4d440704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d44072d60 0x7f4d44167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d44073210 con 0x7f4d4415da30 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d440cd540 con 0x7f4d4406b270 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.203+0000 7f4d4a26f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440b3de0 con 0x7f4d44072d60 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d4406b270 0x7f4d440704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d4880c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d44072d60 0x7f4d44167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d4406b270 0x7f4d440704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47952/0 (socket says 172.21.15.143:47952) 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 -- 172.21.15.143:0/3844791006 learned_addr learned my addr 172.21.15.143:0/3844791006 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 -- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d44072d60 msgr2=0x7f4d44167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:11.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 --2- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d44072d60 0x7f4d44167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 -- 172.21.15.143:0/3844791006 >> v1:172.21.15.116:6789/0 conn(0x7f4d4415da30 legacy=0x7f4d4406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d437fe640 1 -- 172.21.15.143:0/3844791006 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4415de10 con 0x7f4d4406b270 2024-02-18T11:47:11.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.204+0000 7f4d4880c640 1 --2- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d44072d60 0x7f4d44167550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:11.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d437fe640 1 --2- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d440704a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009870 tx=0x7f4d3802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6663e730b8631e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d42ffd640 1 -- 172.21.15.143:0/3844791006 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38030840 con 0x7f4d4406b270 2024-02-18T11:47:11.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d42ffd640 1 -- 172.21.15.143:0/3844791006 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380309a0 con 0x7f4d4406b270 2024-02-18T11:47:11.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d42ffd640 1 -- 172.21.15.143:0/3844791006 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38030d30 con 0x7f4d4406b270 2024-02-18T11:47:11.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 msgr2=0x7f4d440704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d440704a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009870 tx=0x7f4d3802b980 comp rx=0 tx=0).stop 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3844791006 shutdown_connections 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d44072d60 0x7f4d44167550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3844791006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d440704a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3844791006 >> 172.21.15.143:0/3844791006 conn(0x7f4d440b23f0 msgr2=0x7f4d440b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3844791006 shutdown_connections 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.205+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3844791006 wait complete. 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 Processor -- start 2024-02-18T11:47:11.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 -- start start 2024-02-18T11:47:11.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d441f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d44072d60 0x7f4d44160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d4415da30 0x7f4d44143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d440cad60 con 0x7f4d44072d60 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44160d60 con 0x7f4d4406b270 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.207+0000 7f4d4a26f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d441e6000 con 0x7f4d4415da30 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d441f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d4880c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d4415da30 0x7f4d44143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d441f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47962/0 (socket says 172.21.15.143:47962) 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d4880c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d4415da30 0x7f4d44143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39184/0 (socket says 172.21.15.143:39184) 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 -- 172.21.15.143:0/3832586687 learned_addr learned my addr 172.21.15.143:0/3832586687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d437fe640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d44072d60 0x7f4d44160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 -- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d4415da30 msgr2=0x7f4d44143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d4415da30 0x7f4d44143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 -- 172.21.15.143:0/3832586687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d44072d60 msgr2=0x7f4d44160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d44072d60 0x7f4d44160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 -- 172.21.15.143:0/3832586687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c009a70 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d43fff640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d441f73c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c400 tx=0x7f4d3400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340052d0 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d34005450 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d380094d0 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34007260 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d441e6250 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4d340073c0 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.208+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4d34003d30 con 0x7f4d4406b270 2024-02-18T11:47:11.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.209+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d440caac0 con 0x7f4d4406b270 2024-02-18T11:47:11.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.211+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4d440caac0 con 0x7f4d4406b270 2024-02-18T11:47:11.218 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.225+0000 7ff775241040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:47:11.263 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.270+0000 7ff775241040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:47:11.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.308+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "391fd049-5adf-469c-a031-e1d2cf36f723", "id": 2} v 0) v1 -- 0x7f4d440caea0 con 0x7f4d4406b270 2024-02-18T11:47:11.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.311+0000 7f4d40ff9640 1 -- 172.21.15.143:0/3832586687 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "391fd049-5adf-469c-a031-e1d2cf36f723", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4d34020060 con 0x7f4d4406b270 2024-02-18T11:47:11.304 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-02-18T11:47:11.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 msgr2=0x7f4d441f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d441f73c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c400 tx=0x7f4d3400c800 comp rx=0 tx=0).stop 2024-02-18T11:47:11.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 shutdown_connections 2024-02-18T11:47:11.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d4415da30 0x7f4d44143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d44072d60 0x7f4d44160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 --2- 172.21.15.143:0/3832586687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d4406b270 0x7f4d441f73c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 >> 172.21.15.143:0/3832586687 conn(0x7f4d440b23f0 msgr2=0x7f4d44070320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 shutdown_connections 2024-02-18T11:47:11.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.312+0000 7f4d4a26f640 1 -- 172.21.15.143:0/3832586687 wait complete. 2024-02-18T11:47:11.315 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new 755e4fc9-1965-4e13-9955-68bc4ff41573 3 2024-02-18T11:47:11.317 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.324+0000 7ff775241040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:47:11.359 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.366+0000 7ff775241040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:47:11.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.423+0000 7f105d8ff640 1 Processor -- start 2024-02-18T11:47:11.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.423+0000 7f105d8ff640 1 -- start start 2024-02-18T11:47:11.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f105d8ff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105815e6b0 0x7f10580ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f105d8ff640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105815eb40 con 0x7f105815dbc0 2024-02-18T11:47:11.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f105d8ff640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10580b2db0 con 0x7f105806ff70 2024-02-18T11:47:11.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f105d8ff640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580b3090 con 0x7f105815e6b0 2024-02-18T11:47:11.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f10567fc640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f105806ff70 0x7f10580ce0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47192/0 (socket says 172.21.15.143:47192) 2024-02-18T11:47:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f10567fc640 1 -- 172.21.15.143:0/78530722 learned_addr learned my addr 172.21.15.143:0/78530722 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-02-18T11:47:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f10577fe640 1 --2- 172.21.15.143:0/78530722 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105815e6b0 0x7f10580ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f1055ffb640 1 -- 172.21.15.143:0/78530722 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027129674 0 0) 0x7f10580b2db0 con 0x7f105806ff70 2024-02-18T11:47:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f10577fe640 1 -- 172.21.15.143:0/78530722 >> v1:172.21.15.143:6789/0 conn(0x7f105806ff70 legacy=0x7f10580ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f10577fe640 1 -- 172.21.15.143:0/78530722 >> v1:172.21.15.116:6789/0 conn(0x7f105815dbc0 legacy=0x7f105815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.424+0000 7f10577fe640 1 -- 172.21.15.143:0/78530722 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580b3370 con 0x7f105815e6b0 2024-02-18T11:47:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.425+0000 7f10577fe640 1 --2- 172.21.15.143:0/78530722 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105815e6b0 0x7f10580ce800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1050009dc0 tx=0x7f105002cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1de515a15e4a4580 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.425+0000 7f1055ffb640 1 -- 172.21.15.143:0/78530722 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105002dd50 con 0x7f105815e6b0 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.425+0000 7f1055ffb640 1 -- 172.21.15.143:0/78530722 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050035440 con 0x7f105815e6b0 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.425+0000 7f1055ffb640 1 -- 172.21.15.143:0/78530722 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500357d0 con 0x7f105815e6b0 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 -- 172.21.15.143:0/78530722 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105815e6b0 msgr2=0x7f10580ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 --2- 172.21.15.143:0/78530722 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105815e6b0 0x7f10580ce800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1050009dc0 tx=0x7f105002cfa0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 -- 172.21.15.143:0/78530722 shutdown_connections 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 --2- 172.21.15.143:0/78530722 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105815e6b0 0x7f10580ce800 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 -- 172.21.15.143:0/78530722 >> 172.21.15.143:0/78530722 conn(0x7f10580cc820 msgr2=0x7f10580ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 -- 172.21.15.143:0/78530722 shutdown_connections 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.426+0000 7f105d8ff640 1 -- 172.21.15.143:0/78530722 wait complete. 2024-02-18T11:47:11.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.427+0000 7f105d8ff640 1 Processor -- start 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.427+0000 7f105d8ff640 1 -- start start 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f105d8ff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 0x7f105815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f105d8ff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105815dbc0 0x7f10581ef830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f105d8ff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f105815e6b0 0x7f1058156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f105d8ff640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105815a5f0 con 0x7f105815dbc0 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f105d8ff640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105815b3e0 con 0x7f105815e6b0 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f105d8ff640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105815d640 con 0x7f105806ff70 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f1056ffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 0x7f105815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10577fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f105815e6b0 0x7f1058156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f1056ffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 0x7f105815d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39200/0 (socket says 172.21.15.143:39200) 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f1056ffd640 1 -- 172.21.15.143:0/2121301407 learned_addr learned my addr 172.21.15.143:0/2121301407 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10567fc640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105815dbc0 0x7f10581ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10577fe640 1 -- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 msgr2=0x7f105815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10577fe640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 0x7f105815d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10577fe640 1 -- 172.21.15.143:0/2121301407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105815dbc0 msgr2=0x7f10581ef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10577fe640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105815dbc0 0x7f10581ef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f10577fe640 1 -- 172.21.15.143:0/2121301407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1050009a70 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.428+0000 7f1056ffd640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 0x7f105815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f10577fe640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f105815e6b0 0x7f1058156cb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1050034000 tx=0x7f1050004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105003c420 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050040070 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105815a7d0 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105003c7a0 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105815ad90 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1050003d20 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.429+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1050004710 con 0x7f105815e6b0 2024-02-18T11:47:11.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.430+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105815b550 con 0x7f105815e6b0 2024-02-18T11:47:11.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.432+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1050042070 con 0x7f105815e6b0 2024-02-18T11:47:11.473 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.480+0000 7ff775241040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:47:11.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.504+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "755e4fc9-1965-4e13-9955-68bc4ff41573", "id": 3} v 0) v1 -- 0x7f1058074520 con 0x7f105815e6b0 2024-02-18T11:47:11.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.508+0000 7f103ffff640 1 -- 172.21.15.143:0/2121301407 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "755e4fc9-1965-4e13-9955-68bc4ff41573", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f105003a070 con 0x7f105815e6b0 2024-02-18T11:47:11.501 INFO:teuthology.orchestra.run.smithi143.stdout:3 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f105815e6b0 msgr2=0x7f1058156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f105815e6b0 0x7f1058156cb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1050034000 tx=0x7f1050004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 shutdown_connections 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f105815e6b0 0x7f1058156cb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f105815dbc0 0x7f10581ef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 --2- 172.21.15.143:0/2121301407 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f105806ff70 0x7f105815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 >> 172.21.15.143:0/2121301407 conn(0x7f10580cc820 msgr2=0x7f10580cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 shutdown_connections 2024-02-18T11:47:11.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.509+0000 7f105d8ff640 1 -- 172.21.15.143:0/2121301407 wait complete. 2024-02-18T11:47:11.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new 429ccd4b-5332-496c-b884-7d88777a186d 4 2024-02-18T11:47:11.543 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.550+0000 7ff775241040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:47:11.589 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.595+0000 7ff775241040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:47:11.612 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.620+0000 7f88086c3040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T11:47:11.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.622+0000 7f5235a39640 1 Processor -- start 2024-02-18T11:47:11.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.622+0000 7f5235a39640 1 -- start start 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.622+0000 7f5235a39640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f523015a870 0x7f52300b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.622+0000 7f5235a39640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300d1020 con 0x7f5230070970 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.622+0000 7f5235a39640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300b5190 con 0x7f52300b4190 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.622+0000 7f5235a39640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300b45f0 con 0x7f523015a870 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.623+0000 7f522effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f523015a870 0x7f52300b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.623+0000 7f522effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f523015a870 0x7f52300b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47978/0 (socket says 172.21.15.143:47978) 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.623+0000 7f522effd640 1 -- 172.21.15.143:0/3953884089 learned_addr learned my addr 172.21.15.143:0/3953884089 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.623+0000 7f522effd640 1 -- 172.21.15.143:0/3953884089 >> v1:172.21.15.143:6789/0 conn(0x7f52300b4190 legacy=0x7f5230070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.623+0000 7f522effd640 1 -- 172.21.15.143:0/3953884089 >> v1:172.21.15.116:6789/0 conn(0x7f5230070970 legacy=0x7f52300ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.623+0000 7f522effd640 1 -- 172.21.15.143:0/3953884089 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300d0d40 con 0x7f523015a870 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f522effd640 1 --2- 172.21.15.143:0/3953884089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523015a870 0x7f52300b5620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f522400bb10 tx=0x7f522402e170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ebefb03fda8f287 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f522dffb640 1 -- 172.21.15.143:0/3953884089 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224036440 con 0x7f523015a870 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f522dffb640 1 -- 172.21.15.143:0/3953884089 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52240365a0 con 0x7f523015a870 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f522dffb640 1 -- 172.21.15.143:0/3953884089 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224036930 con 0x7f523015a870 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 -- 172.21.15.143:0/3953884089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523015a870 msgr2=0x7f52300b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 --2- 172.21.15.143:0/3953884089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523015a870 0x7f52300b5620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f522400bb10 tx=0x7f522402e170 comp rx=0 tx=0).stop 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 -- 172.21.15.143:0/3953884089 shutdown_connections 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 --2- 172.21.15.143:0/3953884089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f523015a870 0x7f52300b5620 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 -- 172.21.15.143:0/3953884089 >> 172.21.15.143:0/3953884089 conn(0x7f52300b23f0 msgr2=0x7f52300b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 -- 172.21.15.143:0/3953884089 shutdown_connections 2024-02-18T11:47:11.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.624+0000 7f5235a39640 1 -- 172.21.15.143:0/3953884089 wait complete. 2024-02-18T11:47:11.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.625+0000 7f5235a39640 1 Processor -- start 2024-02-18T11:47:11.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 -- start start 2024-02-18T11:47:11.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 0x7f52301527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f52300b4190 0x7f523015b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523015a870 0x7f523015bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523015c790 con 0x7f523015a870 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52301532a0 con 0x7f5230070970 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f5235a39640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523015c1d0 con 0x7f52300b4190 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.626+0000 7f522effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 0x7f52301527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 0x7f52301527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47982/0 (socket says 172.21.15.143:47982) 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 -- 172.21.15.143:0/4042258987 learned_addr learned my addr 172.21.15.143:0/4042258987 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522e7fc640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f52300b4190 0x7f523015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522f7fe640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523015a870 0x7f523015bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 -- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f52300b4190 msgr2=0x7f523015b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f52300b4190 0x7f523015b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 -- 172.21.15.143:0/4042258987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523015a870 msgr2=0x7f523015bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523015a870 0x7f523015bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 -- 172.21.15.143:0/4042258987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522400b770 con 0x7f5230070970 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f522effd640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 0x7f52301527f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f522402e1b0 tx=0x7f52240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224034b80 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224034ce0 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224041680 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523015c4b0 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52301ef7c0 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f522403d040 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.627+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f522404abb0 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.628+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5230074520 con 0x7f5230070970 2024-02-18T11:47:11.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.630+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5224068dd0 con 0x7f5230070970 2024-02-18T11:47:11.641 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.648+0000 7ff775241040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:47:11.670 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.677+0000 7f88086c3040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T11:47:11.687 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:11.694+0000 7ff775241040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:47:11.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.702+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "429ccd4b-5332-496c-b884-7d88777a186d", "id": 4} v 0) v1 -- 0x7f52300cee00 con 0x7f5230070970 2024-02-18T11:47:11.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.708+0000 7f5217fff640 1 -- 172.21.15.143:0/4042258987 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "429ccd4b-5332-496c-b884-7d88777a186d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5224057030 con 0x7f5230070970 2024-02-18T11:47:11.701 INFO:teuthology.orchestra.run.smithi143.stdout:4 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 msgr2=0x7f52301527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 0x7f52301527f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f522402e1b0 tx=0x7f52240046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 shutdown_connections 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523015a870 0x7f523015bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f52300b4190 0x7f523015b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 --2- 172.21.15.143:0/4042258987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5230070970 0x7f52301527f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 >> 172.21.15.143:0/4042258987 conn(0x7f52300b23f0 msgr2=0x7f52300b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 shutdown_connections 2024-02-18T11:47:11.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.709+0000 7f5235a39640 1 -- 172.21.15.143:0/4042258987 wait complete. 2024-02-18T11:47:11.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new 25dce560-148e-49a8-a32b-3bcb4b6bd446 5 2024-02-18T11:47:11.718 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.725+0000 7f88086c3040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T11:47:11.776 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.783+0000 7f88086c3040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T11:47:11.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.821+0000 7fe516177640 1 Processor -- start 2024-02-18T11:47:11.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe516177640 1 -- start start 2024-02-18T11:47:11.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe516177640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe510145250 0x7fe510070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe516177640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5100719d0 0x7fe510071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe516177640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5100ceed0 con 0x7fe5100d11a0 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe516177640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5100b1100 con 0x7fe510145250 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe516177640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5100ce2c0 con 0x7fe5100719d0 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe50effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5100719d0 0x7fe510071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe50effd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5100719d0 0x7fe510071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47984/0 (socket says 172.21.15.143:47984) 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.822+0000 7fe50effd640 1 -- 172.21.15.143:0/1769329942 learned_addr learned my addr 172.21.15.143:0/1769329942 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.823+0000 7fe50f7fe640 1 --2- 172.21.15.143:0/1769329942 >> v2:172.21.15.116:3300/0 conn(0x7fe510145250 0x7fe510070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.823+0000 7fe50effd640 1 -- 172.21.15.143:0/1769329942 >> v2:172.21.15.116:3300/0 conn(0x7fe510145250 msgr2=0x7fe510070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.823+0000 7fe50effd640 1 --2- 172.21.15.143:0/1769329942 >> v2:172.21.15.116:3300/0 conn(0x7fe510145250 0x7fe510070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.823+0000 7fe50effd640 1 -- 172.21.15.143:0/1769329942 >> v1:172.21.15.116:6789/0 conn(0x7fe5100d11a0 legacy=0x7fe51015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.823+0000 7fe50effd640 1 -- 172.21.15.143:0/1769329942 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5100ce5a0 con 0x7fe5100719d0 2024-02-18T11:47:11.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.823+0000 7fe50effd640 1 --2- 172.21.15.143:0/1769329942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5100719d0 0x7fe510071120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe510071980 tx=0x7fe50402b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=396509874b19aea server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe50e7fc640 1 -- 172.21.15.143:0/1769329942 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040317e0 con 0x7fe5100719d0 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe50e7fc640 1 -- 172.21.15.143:0/1769329942 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504031940 con 0x7fe5100719d0 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe50e7fc640 1 -- 172.21.15.143:0/1769329942 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504031cd0 con 0x7fe5100719d0 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 -- 172.21.15.143:0/1769329942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5100719d0 msgr2=0x7fe510071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 --2- 172.21.15.143:0/1769329942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5100719d0 0x7fe510071120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe510071980 tx=0x7fe50402b880 comp rx=0 tx=0).stop 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 -- 172.21.15.143:0/1769329942 shutdown_connections 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 --2- 172.21.15.143:0/1769329942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5100719d0 0x7fe510071120 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 --2- 172.21.15.143:0/1769329942 >> v2:172.21.15.116:3300/0 conn(0x7fe510145250 0x7fe510070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 -- 172.21.15.143:0/1769329942 >> 172.21.15.143:0/1769329942 conn(0x7fe51006f460 msgr2=0x7fe51006f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 -- 172.21.15.143:0/1769329942 shutdown_connections 2024-02-18T11:47:11.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.824+0000 7fe516177640 1 -- 172.21.15.143:0/1769329942 wait complete. 2024-02-18T11:47:11.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.825+0000 7fe516177640 1 Processor -- start 2024-02-18T11:47:11.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 -- start start 2024-02-18T11:47:11.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5100719d0 0x7fe5101e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe5100d11a0 0x7fe5101411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 0x7fe5100b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101e2460 con 0x7fe5100719d0 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5101f5530 con 0x7fe510145250 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe516177640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51013eee0 con 0x7fe5100d11a0 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 0x7fe5100b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.826+0000 7fe50effd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe5100d11a0 0x7fe5101411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 0x7fe5100b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47996/0 (socket says 172.21.15.143:47996) 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50effd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe5100d11a0 0x7fe5101411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39218/0 (socket says 172.21.15.143:39218) 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 -- 172.21.15.143:0/1347360207 learned_addr learned my addr 172.21.15.143:0/1347360207 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50f7fe640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5100719d0 0x7fe5101e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 -- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe5100d11a0 msgr2=0x7fe5101411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe5100d11a0 0x7fe5101411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 -- 172.21.15.143:0/1347360207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5100719d0 msgr2=0x7fe5101e1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5100719d0 0x7fe5101e1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 -- 172.21.15.143:0/1347360207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5040094d0 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50f7fe640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5100719d0 0x7fe5101e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe50ffff640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 0x7fe5100b60a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c7a0 tx=0x7fe50800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508003b70 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508003cf0 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5101f4c40 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508003ea0 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe510141fd0 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe50800aa90 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.827+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe508012a40 con 0x7fe510145250 2024-02-18T11:47:11.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.828+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5100cf7a0 con 0x7fe510145250 2024-02-18T11:47:11.825 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.830+0000 7f88086c3040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T11:47:11.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.832+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe508012ca0 con 0x7fe510145250 2024-02-18T11:47:11.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.904+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "25dce560-148e-49a8-a32b-3bcb4b6bd446", "id": 5} v 0) v1 -- 0x7fe510070be0 con 0x7fe510145250 2024-02-18T11:47:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.910+0000 7fe4f7fff640 1 -- 172.21.15.143:0/1347360207 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25dce560-148e-49a8-a32b-3bcb4b6bd446", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe508012e40 con 0x7fe510145250 2024-02-18T11:47:11.903 INFO:teuthology.orchestra.run.smithi143.stdout:5 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 msgr2=0x7fe5100b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 0x7fe5100b60a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c7a0 tx=0x7fe50800cc60 comp rx=0 tx=0).stop 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 shutdown_connections 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe510145250 0x7fe5100b60a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe5100d11a0 0x7fe5101411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 --2- 172.21.15.143:0/1347360207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5100719d0 0x7fe5101e1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 >> 172.21.15.143:0/1347360207 conn(0x7fe51006f460 msgr2=0x7fe5101e7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 shutdown_connections 2024-02-18T11:47:11.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:11.911+0000 7fe516177640 1 -- 172.21.15.143:0/1347360207 wait complete. 2024-02-18T11:47:11.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new 2239dfcb-8d00-4ce1-8f4f-9d366513bafa 6 2024-02-18T11:47:11.945 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:11.953+0000 7f88086c3040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-18T11:47:12.020 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.027+0000 7f88086c3040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T11:47:12.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.053+0000 7f7e8c80b640 1 Processor -- start 2024-02-18T11:47:12.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.053+0000 7f7e8c80b640 1 -- start start 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8c80b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e8815a870 0x7f7e880b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8c80b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e880b4190 0x7f7e880d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8c80b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e88070260 0x7f7e880706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8c80b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e88070c30 con 0x7f7e88070260 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8c80b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e880d0b30 con 0x7f7e880b4190 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8c80b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e880d1020 con 0x7f7e8815a870 2024-02-18T11:47:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e880b4190 0x7f7e880d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8659c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e8815a870 0x7f7e880b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e880b4190 0x7f7e880d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48010/0 (socket says 172.21.15.143:48010) 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 -- 172.21.15.143:0/1286356906 learned_addr learned my addr 172.21.15.143:0/1286356906 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e86d9d640 1 --2- 172.21.15.143:0/1286356906 >> v2:172.21.15.116:3300/0 conn(0x7f7e88070260 0x7f7e880706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 -- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e8815a870 msgr2=0x7f7e880b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 --2- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e8815a870 0x7f7e880b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 -- 172.21.15.143:0/1286356906 >> v2:172.21.15.116:3300/0 conn(0x7f7e88070260 msgr2=0x7f7e880706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 --2- 172.21.15.143:0/1286356906 >> v2:172.21.15.116:3300/0 conn(0x7f7e88070260 0x7f7e880706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e85d9b640 1 -- 172.21.15.143:0/1286356906 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e880b5190 con 0x7f7e880b4190 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e86d9d640 1 --2- 172.21.15.143:0/1286356906 >> v2:172.21.15.116:3300/0 conn(0x7f7e88070260 0x7f7e880706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.054+0000 7f7e8659c640 1 --2- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e8815a870 0x7f7e880b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.055+0000 7f7e85d9b640 1 --2- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e880b4190 0x7f7e880d05f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7e8806f300 tx=0x7f7e7802b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672cb5f2b40961ac server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.055+0000 7f7e8559a640 1 -- 172.21.15.143:0/1286356906 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78031750 con 0x7f7e880b4190 2024-02-18T11:47:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.055+0000 7f7e8559a640 1 -- 172.21.15.143:0/1286356906 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e780318b0 con 0x7f7e880b4190 2024-02-18T11:47:12.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.055+0000 7f7e8559a640 1 -- 172.21.15.143:0/1286356906 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78031c40 con 0x7f7e880b4190 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 -- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e880b4190 msgr2=0x7f7e880d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e880b4190 0x7f7e880d05f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7e8806f300 tx=0x7f7e7802b7f0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 -- 172.21.15.143:0/1286356906 shutdown_connections 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/1286356906 >> v2:172.21.15.116:3300/0 conn(0x7f7e88070260 0x7f7e880706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e880b4190 0x7f7e880d05f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/1286356906 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e8815a870 0x7f7e880b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 -- 172.21.15.143:0/1286356906 >> 172.21.15.143:0/1286356906 conn(0x7f7e880b23f0 msgr2=0x7f7e880b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 -- 172.21.15.143:0/1286356906 shutdown_connections 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.056+0000 7f7e8c80b640 1 -- 172.21.15.143:0/1286356906 wait complete. 2024-02-18T11:47:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.057+0000 7f7e8c80b640 1 Processor -- start 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.057+0000 7f7e8c80b640 1 -- start start 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8c80b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e88070260 0x7f7e880ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8c80b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 0x7f7e88142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8c80b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 0x7f7e88142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8c80b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e88142ad0 con 0x7f7e88070260 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8c80b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e88143660 con 0x7f7e8815a870 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8c80b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e88145c10 con 0x7f7e880b4190 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e85d9b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 0x7f7e88142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 0x7f7e88142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 0x7f7e88142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48018/0 (socket says 172.21.15.143:48018) 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e85d9b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 0x7f7e88142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39242/0 (socket says 172.21.15.143:39242) 2024-02-18T11:47:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 -- 172.21.15.143:0/223934293 learned_addr learned my addr 172.21.15.143:0/223934293 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e8659c640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e88070260 0x7f7e880ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 -- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 msgr2=0x7f7e88142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 0x7f7e88142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 -- 172.21.15.143:0/223934293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e88070260 msgr2=0x7f7e880ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e88070260 0x7f7e880ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e86d9d640 1 -- 172.21.15.143:0/223934293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c00b6a0 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.058+0000 7f7e85d9b640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 0x7f7e88142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e86d9d640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 0x7f7e88142590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e80007ef0 tx=0x7f7e8000d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003b90 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8000f040 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e780094d0 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8000de80 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e88143010 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7e80004030 con 0x7f7e8815a870 2024-02-18T11:47:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.059+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7e80015440 con 0x7f7e8815a870 2024-02-18T11:47:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.060+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e88074520 con 0x7f7e8815a870 2024-02-18T11:47:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.062+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7e80024060 con 0x7f7e8815a870 2024-02-18T11:47:12.066 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.073+0000 7ff775241040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:47:12.070 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.077+0000 7f88086c3040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T11:47:12.124 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.132+0000 7f88086c3040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T11:47:12.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.135+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2239dfcb-8d00-4ce1-8f4f-9d366513bafa", "id": 6} v 0) v1 -- 0x7f7e881437a0 con 0x7f7e8815a870 2024-02-18T11:47:12.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.140+0000 7f7e76ffd640 1 -- 172.21.15.143:0/223934293 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2239dfcb-8d00-4ce1-8f4f-9d366513bafa", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e8001a070 con 0x7f7e8815a870 2024-02-18T11:47:12.133 INFO:teuthology.orchestra.run.smithi143.stdout:6 2024-02-18T11:47:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 msgr2=0x7f7e88142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 0x7f7e88142590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e80007ef0 tx=0x7f7e8000d4a0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 shutdown_connections 2024-02-18T11:47:12.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e8815a870 0x7f7e88142590 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7e880b4190 0x7f7e88142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 --2- 172.21.15.143:0/223934293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e88070260 0x7f7e880ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 >> 172.21.15.143:0/223934293 conn(0x7f7e880b23f0 msgr2=0x7f7e880b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 shutdown_connections 2024-02-18T11:47:12.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.141+0000 7f7e8c80b640 1 -- 172.21.15.143:0/223934293 wait complete. 2024-02-18T11:47:12.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd new ab909074-4daa-42df-b320-e78bf4f1e2dd 7 2024-02-18T11:47:12.175 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.183+0000 7f88086c3040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-18T11:47:12.200 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.207+0000 7ff775241040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:47:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.252+0000 7f838c777640 1 Processor -- start 2024-02-18T11:47:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.252+0000 7f838c777640 1 -- start start 2024-02-18T11:47:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.253+0000 7f838c777640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f838415da30 0x7f83840704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.253+0000 7f838c777640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8384072d60 0x7f838406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.253+0000 7f838c777640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838406b4f0 con 0x7f838406b0a0 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.253+0000 7f838c777640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840cd540 con 0x7f8384072d60 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.253+0000 7f838c777640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83840b3de0 con 0x7f838415da30 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f838415da30 0x7f83840704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f838415da30 0x7f83840704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48028/0 (socket says 172.21.15.143:48028) 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 -- 172.21.15.143:0/3767084814 learned_addr learned my addr 172.21.15.143:0/3767084814 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f8389d12640 1 --2- 172.21.15.143:0/3767084814 >> v2:172.21.15.116:3300/0 conn(0x7f8384072d60 0x7f838406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 -- 172.21.15.143:0/3767084814 >> v2:172.21.15.116:3300/0 conn(0x7f8384072d60 msgr2=0x7f838406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 --2- 172.21.15.143:0/3767084814 >> v2:172.21.15.116:3300/0 conn(0x7f8384072d60 0x7f838406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 -- 172.21.15.143:0/3767084814 >> v1:172.21.15.143:6789/0 conn(0x7f838406b0a0 legacy=0x7f83840d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.254+0000 7f838a513640 1 -- 172.21.15.143:0/3767084814 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83840731a0 con 0x7f838415da30 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838a513640 1 --2- 172.21.15.143:0/3767084814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f83840704a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f83840b5bc0 tx=0x7f838002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e3157e2fa65b76 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f8389511640 1 -- 172.21.15.143:0/3767084814 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380030860 con 0x7f838415da30 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f8389511640 1 -- 172.21.15.143:0/3767084814 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83800309c0 con 0x7f838415da30 2024-02-18T11:47:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f8389511640 1 -- 172.21.15.143:0/3767084814 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380030d50 con 0x7f838415da30 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 -- 172.21.15.143:0/3767084814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 msgr2=0x7f83840704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 --2- 172.21.15.143:0/3767084814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f83840704a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f83840b5bc0 tx=0x7f838002b9a0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 -- 172.21.15.143:0/3767084814 shutdown_connections 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 --2- 172.21.15.143:0/3767084814 >> v2:172.21.15.116:3300/0 conn(0x7f8384072d60 0x7f838406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 --2- 172.21.15.143:0/3767084814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f83840704a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 -- 172.21.15.143:0/3767084814 >> 172.21.15.143:0/3767084814 conn(0x7f83840b23f0 msgr2=0x7f83840b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:12.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 -- 172.21.15.143:0/3767084814 shutdown_connections 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.255+0000 7f838c777640 1 -- 172.21.15.143:0/3767084814 wait complete. 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 Processor -- start 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 -- start start 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f838406b0a0 0x7f83841f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 0x7f838415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f838415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83841f7a30 con 0x7f838406b0a0 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83841f6c70 con 0x7f838415da30 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838c777640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83841f6f50 con 0x7f8384072d60 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f8389d12640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 0x7f838415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838ad14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f838415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f8389d12640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 0x7f838415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39258/0 (socket says 172.21.15.143:39258) 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.257+0000 7f838ad14640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f838415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48032/0 (socket says 172.21.15.143:48032) 2024-02-18T11:47:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 -- 172.21.15.143:0/3427524632 learned_addr learned my addr 172.21.15.143:0/3427524632 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f838a513640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f838406b0a0 0x7f83841f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 -- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 msgr2=0x7f838415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f838415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 -- 172.21.15.143:0/3427524632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f838406b0a0 msgr2=0x7f83841f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f838406b0a0 0x7f83841f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 -- 172.21.15.143:0/3427524632 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83800094d0 con 0x7f8384072d60 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f838ad14640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f838415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f8389d12640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 0x7f838415a6a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f837400e870 tx=0x7f837400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83740094f0 con 0x7f8384072d60 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400c940 con 0x7f8384072d60 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83841f7230 con 0x7f8384072d60 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400ccf0 con 0x7f8384072d60 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838415b210 con 0x7f8384072d60 2024-02-18T11:47:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8374005990 con 0x7f8384072d60 2024-02-18T11:47:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.258+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f8374006840 con 0x7f8384072d60 2024-02-18T11:47:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.259+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8384074520 con 0x7f8384072d60 2024-02-18T11:47:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.261+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8374021060 con 0x7f8384072d60 2024-02-18T11:47:12.307 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.313+0000 7ff775241040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:47:12.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.334+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ab909074-4daa-42df-b320-e78bf4f1e2dd", "id": 7} v 0) v1 -- 0x7f83840b12e0 con 0x7f8384072d60 2024-02-18T11:47:12.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.339+0000 7f837affd640 1 -- 172.21.15.143:0/3427524632 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab909074-4daa-42df-b320-e78bf4f1e2dd", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8374017070 con 0x7f8384072d60 2024-02-18T11:47:12.332 INFO:teuthology.orchestra.run.smithi143.stdout:7 2024-02-18T11:47:12.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 msgr2=0x7f838415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 0x7f838415a6a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f837400e870 tx=0x7f837400ed30 comp rx=0 tx=0).stop 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 shutdown_connections 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f838415da30 0x7f838415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8384072d60 0x7f838415a6a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 --2- 172.21.15.143:0/3427524632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f838406b0a0 0x7f83841f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 >> 172.21.15.143:0/3427524632 conn(0x7f83840b23f0 msgr2=0x7f83840ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 shutdown_connections 2024-02-18T11:47:12.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-02-18T11:47:12.340+0000 7f838c777640 1 -- 172.21.15.143:0/3427524632 wait complete. 2024-02-18T11:47:12.344 INFO:tasks.ceph.osd.0:Restarting daemon 2024-02-18T11:47:12.344 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-02-18T11:47:12.346 INFO:tasks.ceph.osd.0:Started 2024-02-18T11:47:12.346 INFO:tasks.ceph.osd.1:Restarting daemon 2024-02-18T11:47:12.346 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-02-18T11:47:12.348 INFO:tasks.ceph.osd.1:Started 2024-02-18T11:47:12.348 INFO:tasks.ceph.osd.2:Restarting daemon 2024-02-18T11:47:12.349 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-02-18T11:47:12.350 INFO:tasks.ceph.osd.2:Started 2024-02-18T11:47:12.351 INFO:tasks.ceph.osd.3:Restarting daemon 2024-02-18T11:47:12.351 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-02-18T11:47:12.353 INFO:tasks.ceph.osd.3:Started 2024-02-18T11:47:12.353 INFO:tasks.ceph.osd.4:Restarting daemon 2024-02-18T11:47:12.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-02-18T11:47:12.354 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.361+0000 7ff775241040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:47:12.386 INFO:tasks.ceph.osd.4:Started 2024-02-18T11:47:12.386 INFO:tasks.ceph.osd.5:Restarting daemon 2024-02-18T11:47:12.386 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-02-18T11:47:12.388 INFO:tasks.ceph.osd.5:Started 2024-02-18T11:47:12.389 INFO:tasks.ceph.osd.6:Restarting daemon 2024-02-18T11:47:12.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-02-18T11:47:12.391 INFO:tasks.ceph.osd.6:Started 2024-02-18T11:47:12.391 INFO:tasks.ceph.osd.7:Restarting daemon 2024-02-18T11:47:12.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-02-18T11:47:12.393 INFO:tasks.ceph.osd.7:Started 2024-02-18T11:47:12.394 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:12.487 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.494+0000 7ff775241040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:47:12.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.519+0000 7f9038876640 1 Processor -- start 2024-02-18T11:47:12.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.519+0000 7f9038876640 1 -- start start 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f9038876640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9034161850 0x7f90340b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f9038876640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90340d3730 con 0x7f903415dc60 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f9038876640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90340cf250 con 0x7f90340d1080 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f9038876640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90340ce9b0 con 0x7f9034161850 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f903259c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f903415dc60 0x7f90340d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55146/0 (socket says 172.21.15.116:55146) 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f903259c640 1 -- 172.21.15.116:0/4131740184 learned_addr learned my addr 172.21.15.116:0/4131740184 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:12.514 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.520+0000 7f9032d9d640 1 --2- 172.21.15.116:0/4131740184 >> v2:172.21.15.143:3300/0 conn(0x7f9034161850 0x7f90340b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.521+0000 7f9031d9b640 1 -- 172.21.15.116:0/4131740184 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914239975 0 0) 0x7f90340d3730 con 0x7f903415dc60 2024-02-18T11:47:12.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.521+0000 7f9031d9b640 1 -- 172.21.15.116:0/4131740184 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9020003520 con 0x7f903415dc60 2024-02-18T11:47:12.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.521+0000 7f9032d9d640 1 -- 172.21.15.116:0/4131740184 >> v1:172.21.15.143:6789/0 conn(0x7f90340d1080 legacy=0x7f90340d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.521+0000 7f9032d9d640 1 -- 172.21.15.116:0/4131740184 >> v1:172.21.15.116:6789/0 conn(0x7f903415dc60 legacy=0x7f90340d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.515 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.521+0000 7f9032d9d640 1 -- 172.21.15.116:0/4131740184 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90340cec90 con 0x7f9034161850 2024-02-18T11:47:12.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.522+0000 7f9032d9d640 1 --2- 172.21.15.116:0/4131740184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9034161850 0x7f90340b6c10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f90340d38c0 tx=0x7f902c02bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d758d36133db6c11 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:12.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.522+0000 7f9031d9b640 1 -- 172.21.15.116:0/4131740184 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c02cce0 con 0x7f9034161850 2024-02-18T11:47:12.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.522+0000 7f9031d9b640 1 -- 172.21.15.116:0/4131740184 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c02ce40 con 0x7f9034161850 2024-02-18T11:47:12.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.522+0000 7f9031d9b640 1 -- 172.21.15.116:0/4131740184 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c034660 con 0x7f9034161850 2024-02-18T11:47:12.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 -- 172.21.15.116:0/4131740184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9034161850 msgr2=0x7f90340b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 --2- 172.21.15.116:0/4131740184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9034161850 0x7f90340b6c10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f90340d38c0 tx=0x7f902c02bf30 comp rx=0 tx=0).stop 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 -- 172.21.15.116:0/4131740184 shutdown_connections 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 --2- 172.21.15.116:0/4131740184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9034161850 0x7f90340b6c10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 -- 172.21.15.116:0/4131740184 >> 172.21.15.116:0/4131740184 conn(0x7f90340b5910 msgr2=0x7f90340b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 -- 172.21.15.116:0/4131740184 shutdown_connections 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.523+0000 7f9038876640 1 -- 172.21.15.116:0/4131740184 wait complete. 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.524+0000 7f9038876640 1 Processor -- start 2024-02-18T11:47:12.517 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.524+0000 7f9038876640 1 -- start start 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9038876640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 0x7f90340cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9038876640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903415dc60 0x7f90340cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9038876640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9034161850 0x7f90340d3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9038876640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9034155940 con 0x7f90340d1080 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9038876640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9034155c20 con 0x7f903415dc60 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9038876640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9034155f00 con 0x7f9034161850 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 0x7f90340cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 0x7f90340cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33538/0 (socket says 172.21.15.116:33538) 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 -- 172.21.15.116:0/3486237925 learned_addr learned my addr 172.21.15.116:0/3486237925 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f902bfff640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903415dc60 0x7f90340cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f9032d9d640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9034161850 0x7f90340d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 -- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9034161850 msgr2=0x7f90340d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9034161850 0x7f90340d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 -- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903415dc60 msgr2=0x7f90340cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903415dc60 0x7f90340cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 -- 172.21.15.116:0/3486237925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c009a70 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.525+0000 7f903259c640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 0x7f90340cb730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f902400c500 tx=0x7f902400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024005ce0 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9024005e40 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9024007330 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903415e1c0 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90341564c0 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9024007490 con 0x7f90340d1080 2024-02-18T11:47:12.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.526+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9024012bc0 con 0x7f90340d1080 2024-02-18T11:47:12.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.527+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9034067840 con 0x7f90340d1080 2024-02-18T11:47:12.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.529+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9024017070 con 0x7f90340d1080 2024-02-18T11:47:12.596 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.604+0000 7f88086c3040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T11:47:12.602 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.609+0000 7ff775241040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:47:12.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.610+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9034074520 con 0x7f90340d1080 2024-02-18T11:47:12.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.610+0000 7f902affd640 1 -- 172.21.15.116:0/3486237925 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f9024012550 con 0x7f90340d1080 2024-02-18T11:47:12.604 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:12.604 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":10,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:12.335646+0000","last_up_change":"0.000000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:12.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 msgr2=0x7f90340cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 0x7f90340cb730 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f902400c500 tx=0x7f902400c9c0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 shutdown_connections 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9034161850 0x7f90340d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903415dc60 0x7f90340cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 --2- 172.21.15.116:0/3486237925 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90340d1080 0x7f90340cb730 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 >> 172.21.15.116:0/3486237925 conn(0x7f90340b5910 msgr2=0x7f90340b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 shutdown_connections 2024-02-18T11:47:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:12.611+0000 7f9038876640 1 -- 172.21.15.116:0/3486237925 wait complete. 2024-02-18T11:47:12.615 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-02-18T11:47:12.615 INFO:tasks.ceph:Waiting for OSDs to come up 2024-02-18T11:47:12.730 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.738+0000 7f88086c3040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T11:47:12.761 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.767+0000 7ff775241040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:47:12.803 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.810+0000 7ff775241040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:47:12.860 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.868+0000 7f88086c3040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T11:47:12.907 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:12.914+0000 7f88086c3040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T11:47:12.959 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:12.965+0000 7ff775241040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:47:12.960 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T11:47:12.967+0000 7f9b39ab1640 -1 Falling back to public interface 2024-02-18T11:47:12.963 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T11:47:12.970+0000 7f348f80f640 -1 Falling back to public interface 2024-02-18T11:47:12.971 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T11:47:12.978+0000 7f5c5629d640 -1 Falling back to public interface 2024-02-18T11:47:12.972 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T11:47:12.978+0000 7f6281411640 -1 Falling back to public interface 2024-02-18T11:47:12.994 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T11:47:13.001+0000 7f2d67126640 -1 Falling back to public interface 2024-02-18T11:47:12.994 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T11:47:13.001+0000 7fed5600d640 -1 Falling back to public interface 2024-02-18T11:47:12.995 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T11:47:13.002+0000 7fed548fe640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2024-02-18T11:47:12.995 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T11:47:13.002+0000 7f2d660d5640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2024-02-18T11:47:13.019 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T11:47:13.026+0000 7ff39c2d9640 -1 Falling back to public interface 2024-02-18T11:47:13.021 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:13.029+0000 7f88086c3040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:47:13.023 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T11:47:13.030+0000 7fe4ffc15640 -1 Falling back to public interface 2024-02-18T11:47:13.122 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:13.129+0000 7f88086c3040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T11:47:13.275 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:13.282+0000 7f88086c3040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T11:47:13.319 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:13.326+0000 7f88086c3040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T11:47:13.421 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:13.471 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:13.478+0000 7f88086c3040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T11:47:13.511 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.517+0000 7f6b657d0640 1 Processor -- start 2024-02-18T11:47:13.511 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.517+0000 7f6b657d0640 1 -- start start 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b657d0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b6006ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b657d0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600cb9e0 con 0x7f6b600cb3c0 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b657d0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6015c920 con 0x7f6b6006b0a0 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b657d0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600cdf60 con 0x7f6b600cdb80 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b5f7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6b600cb3c0 0x7f6b6015f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55160/0 (socket says 172.21.15.116:55160) 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b5f7fe640 1 -- 172.21.15.116:0/3929139873 learned_addr learned my addr 172.21.15.116:0/3929139873 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:13.512 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.518+0000 7f6b5effd640 1 --2- 172.21.15.116:0/3929139873 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b6006ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:13.513 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.519+0000 7f6b5dffb640 1 -- 172.21.15.116:0/3929139873 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562784775 0 0) 0x7f6b600cb9e0 con 0x7f6b600cb3c0 2024-02-18T11:47:13.513 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.519+0000 7f6b5dffb640 1 -- 172.21.15.116:0/3929139873 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b48003520 con 0x7f6b600cb3c0 2024-02-18T11:47:13.513 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.519+0000 7f6b5effd640 1 -- 172.21.15.116:0/3929139873 >> v1:172.21.15.143:6789/0 conn(0x7f6b6006b0a0 legacy=0x7f6b600cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:13.513 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.519+0000 7f6b5effd640 1 -- 172.21.15.116:0/3929139873 >> v1:172.21.15.116:6789/0 conn(0x7f6b600cb3c0 legacy=0x7f6b6015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:13.513 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.519+0000 7f6b5effd640 1 -- 172.21.15.116:0/3929139873 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6006b4e0 con 0x7f6b600cdb80 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.519+0000 7f6b5effd640 1 --2- 172.21.15.116:0/3929139873 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b6006ab60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b500097d0 tx=0x7f6b5002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8d0005dbc77180 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b5dffb640 1 -- 172.21.15.116:0/3929139873 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50031890 con 0x7f6b600cdb80 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b5dffb640 1 -- 172.21.15.116:0/3929139873 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b500319f0 con 0x7f6b600cdb80 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b5dffb640 1 -- 172.21.15.116:0/3929139873 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50031d80 con 0x7f6b600cdb80 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 -- 172.21.15.116:0/3929139873 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 msgr2=0x7f6b6006ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 --2- 172.21.15.116:0/3929139873 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b6006ab60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6b500097d0 tx=0x7f6b5002b930 comp rx=0 tx=0).stop 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 -- 172.21.15.116:0/3929139873 shutdown_connections 2024-02-18T11:47:13.514 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 --2- 172.21.15.116:0/3929139873 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b6006ab60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.515 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 -- 172.21.15.116:0/3929139873 >> 172.21.15.116:0/3929139873 conn(0x7f6b600b2330 msgr2=0x7f6b600b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:13.515 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 -- 172.21.15.116:0/3929139873 shutdown_connections 2024-02-18T11:47:13.515 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.520+0000 7f6b657d0640 1 -- 172.21.15.116:0/3929139873 wait complete. 2024-02-18T11:47:13.517 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 Processor -- start 2024-02-18T11:47:13.517 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 -- start start 2024-02-18T11:47:13.517 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 0x7f6b60145160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:13.517 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b600cb3c0 0x7f6b60150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:13.517 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b601458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:13.517 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601456a0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60148740 con 0x7f6b600cb3c0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.522+0000 7f6b657d0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60151fc0 con 0x7f6b600cdb80 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 0x7f6b60145160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 0x7f6b60145160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33554/0 (socket says 172.21.15.116:33554) 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 -- 172.21.15.116:0/1428282039 learned_addr learned my addr 172.21.15.116:0/1428282039 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5f7fe640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b601458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5e7fc640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b600cb3c0 0x7f6b60150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 -- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 msgr2=0x7f6b601458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b601458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 -- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b600cb3c0 msgr2=0x7f6b60150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b600cb3c0 0x7f6b60150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 -- 172.21.15.116:0/1428282039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50009400 con 0x7f6b6006b0a0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b5effd640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 0x7f6b60145160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b50034040 tx=0x7f6b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:13.518 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5003ad20 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b500443f0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b601522a0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500447a0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b600cf3e0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6b50044c90 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.523+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6b5004ddb0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.519 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.524+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b601527c0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.520 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.526+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6b601527c0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.624 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.631+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b600cf5d0 con 0x7f6b6006b0a0 2024-02-18T11:47:13.625 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.632+0000 7f6b477fe640 1 -- 172.21.15.116:0/1428282039 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f6b50035020 con 0x7f6b6006b0a0 2024-02-18T11:47:13.625 INFO:teuthology.misc.health.smithi116.stdout: 2024-02-18T11:47:13.625 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":10,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:12.335646+0000","last_up_change":"0.000000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 msgr2=0x7f6b60145160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 0x7f6b60145160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6b50034040 tx=0x7f6b500046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 shutdown_connections 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6b600cdb80 0x7f6b601458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b600cb3c0 0x7f6b60150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 --2- 172.21.15.116:0/1428282039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b6006b0a0 0x7f6b60145160 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 >> 172.21.15.116:0/1428282039 conn(0x7f6b600b2330 msgr2=0x7f6b6015fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 shutdown_connections 2024-02-18T11:47:13.627 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:13.633+0000 7f6b657d0640 1 -- 172.21.15.116:0/1428282039 wait complete. 2024-02-18T11:47:13.636 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-02-18T11:47:13.822 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:13.829+0000 7ff775241040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:47:14.095 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:14.102+0000 7ff775241040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:47:14.147 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:14.153+0000 7ff775241040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:47:14.260 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:14.267+0000 7ff775241040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:47:14.298 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:14.305+0000 7f88086c3040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T11:47:14.577 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:14.584+0000 7f88086c3040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T11:47:14.628 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:14.635+0000 7f88086c3040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T11:47:14.745 INFO:tasks.ceph.mgr.x.smithi143.stderr:2024-02-18T11:47:14.752+0000 7f88086c3040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T11:47:15.286 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:15.293+0000 7ff715ed1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-18T11:47:15.286 INFO:tasks.ceph.mgr.y.smithi116.stderr:2024-02-18T11:47:15.293+0000 7ff715ed1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-18T11:47:16.465 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T11:47:16.473+0000 7fed5600d640 -1 osd.6 0 log_to_monitors true 2024-02-18T11:47:16.494 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T11:47:16.501+0000 7f2d67126640 -1 osd.4 0 log_to_monitors true 2024-02-18T11:47:16.509 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T11:47:16.516+0000 7f9b39ab1640 -1 osd.0 0 log_to_monitors true 2024-02-18T11:47:16.523 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T11:47:16.530+0000 7f6281411640 -1 osd.2 0 log_to_monitors true 2024-02-18T11:47:16.525 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T11:47:16.532+0000 7f5c5629d640 -1 osd.3 0 log_to_monitors true 2024-02-18T11:47:16.531 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T11:47:16.538+0000 7f348f80f640 -1 osd.1 0 log_to_monitors true 2024-02-18T11:47:16.541 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T11:47:16.549+0000 7ff39c2d9640 -1 osd.7 0 log_to_monitors true 2024-02-18T11:47:16.556 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T11:47:16.564+0000 7fe4ffc15640 -1 osd.5 0 log_to_monitors true 2024-02-18T11:47:20.386 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T11:47:20.391+0000 7f5c52233640 -1 osd.3 0 waiting for initial osdmap 2024-02-18T11:47:20.387 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T11:47:20.394+0000 7fe4fb5b7640 -1 osd.5 0 waiting for initial osdmap 2024-02-18T11:47:20.400 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T11:47:20.408+0000 7fed520f9640 -1 osd.6 0 waiting for initial osdmap 2024-02-18T11:47:20.402 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T11:47:20.409+0000 7f5c4a053640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.402 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T11:47:20.410+0000 7fe4f33d7640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.403 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T11:47:20.410+0000 7f9b35a47640 -1 osd.0 0 waiting for initial osdmap 2024-02-18T11:47:20.407 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T11:47:20.415+0000 7f2d630bc640 -1 osd.4 0 waiting for initial osdmap 2024-02-18T11:47:20.412 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T11:47:20.419+0000 7f627ce04640 -1 osd.2 0 waiting for initial osdmap 2024-02-18T11:47:20.416 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T11:47:20.424+0000 7fed49705640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.420 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T11:47:20.428+0000 7f2d5aedc640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.423 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T11:47:20.430+0000 7f348baa9640 -1 osd.1 0 waiting for initial osdmap 2024-02-18T11:47:20.424 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T11:47:20.431+0000 7f9b2d867640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.428 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T11:47:20.435+0000 7f6274c24640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.438 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T11:47:20.445+0000 7f34830b5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.440 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T11:47:20.448+0000 7ff39826f640 -1 osd.7 0 waiting for initial osdmap 2024-02-18T11:47:20.443 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:20.455 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T11:47:20.463+0000 7ff39008f640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-18T11:47:20.531 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 Processor -- start 2024-02-18T11:47:20.531 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 -- start start 2024-02-18T11:47:20.533 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff92815dd40 0x7ff92815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff9280ccbb0 0x7ff9280cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff92806e3e0 0x7ff92806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92806edd0 con 0x7ff92815dd40 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92815e120 con 0x7ff9280ccbb0 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.538+0000 7ff92e215640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280716c0 con 0x7ff92806e3e0 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff92815dd40 0x7ff92815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff92815dd40 0x7ff92815adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35906/0 (socket says 172.21.15.116:35906) 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 -- 172.21.15.116:0/1272799861 learned_addr learned my addr 172.21.15.116:0/1272799861 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff926ffd640 1 --2- 172.21.15.116:0/1272799861 >> v2:172.21.15.143:3300/0 conn(0x7ff9280ccbb0 0x7ff9280cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff927fff640 1 --2- 172.21.15.116:0/1272799861 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff92806e3e0 0x7ff92806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 -- 172.21.15.116:0/1272799861 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff92806e3e0 msgr2=0x7ff92806e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 --2- 172.21.15.116:0/1272799861 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff92806e3e0 0x7ff92806e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.534 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 -- 172.21.15.116:0/1272799861 >> v2:172.21.15.143:3300/0 conn(0x7ff9280ccbb0 msgr2=0x7ff9280cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 --2- 172.21.15.116:0/1272799861 >> v2:172.21.15.143:3300/0 conn(0x7ff9280ccbb0 0x7ff9280cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.539+0000 7ff9277fe640 1 -- 172.21.15.116:0/1272799861 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280719a0 con 0x7ff92815dd40 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff9277fe640 1 --2- 172.21.15.116:0/1272799861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92815dd40 0x7ff92815adc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9280cf640 tx=0x7ff91802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ec674a2f330e4e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff9267fc640 1 -- 172.21.15.116:0/1272799861 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180317e0 con 0x7ff92815dd40 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff9267fc640 1 -- 172.21.15.116:0/1272799861 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff918031940 con 0x7ff92815dd40 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff9267fc640 1 -- 172.21.15.116:0/1272799861 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918031cd0 con 0x7ff92815dd40 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 -- 172.21.15.116:0/1272799861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92815dd40 msgr2=0x7ff92815adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 --2- 172.21.15.116:0/1272799861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92815dd40 0x7ff92815adc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9280cf640 tx=0x7ff91802b880 comp rx=0 tx=0).stop 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 -- 172.21.15.116:0/1272799861 shutdown_connections 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 --2- 172.21.15.116:0/1272799861 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff92806e3e0 0x7ff92806e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 --2- 172.21.15.116:0/1272799861 >> v2:172.21.15.143:3300/0 conn(0x7ff9280ccbb0 0x7ff9280cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 --2- 172.21.15.116:0/1272799861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92815dd40 0x7ff92815adc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 -- 172.21.15.116:0/1272799861 >> 172.21.15.116:0/1272799861 conn(0x7ff92806ab60 msgr2=0x7ff92806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 -- 172.21.15.116:0/1272799861 shutdown_connections 2024-02-18T11:47:20.535 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.540+0000 7ff92e215640 1 -- 172.21.15.116:0/1272799861 wait complete. 2024-02-18T11:47:20.538 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.541+0000 7ff92e215640 1 Processor -- start 2024-02-18T11:47:20.538 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 -- start start 2024-02-18T11:47:20.538 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 0x7ff9280ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.538 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff9280ccbb0 0x7ff92813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff92815dd40 0x7ff92806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280ae590 con 0x7ff92806e3e0 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280aaa50 con 0x7ff92815dd40 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff92e215640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280aad30 con 0x7ff9280ccbb0 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff9277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 0x7ff9280ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff9277fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 0x7ff9280ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35908/0 (socket says 172.21.15.116:35908) 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.542+0000 7ff9277fe640 1 -- 172.21.15.116:0/3279120699 learned_addr learned my addr 172.21.15.116:0/3279120699 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff926ffd640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff9280ccbb0 0x7ff92813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff927fff640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff92815dd40 0x7ff92806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff9277fe640 1 -- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff9280ccbb0 msgr2=0x7ff92813f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff9277fe640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff9280ccbb0 0x7ff92813f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff9277fe640 1 -- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff92815dd40 msgr2=0x7ff92806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff9277fe640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff92815dd40 0x7ff92806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.539 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff9277fe640 1 -- 172.21.15.116:0/3279120699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180094d0 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff926ffd640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff9280ccbb0 0x7ff92813f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff9277fe640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 0x7ff9280ae050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff9180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918032880 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9180329e0 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9280ab010 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918032d90 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.543+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9280ab5d0 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.544+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92804ef20 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.544+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 4) v1 ==== 90050+0+0 (secure 0 0 0) 0x7ff91804e450 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.545+0000 7ff90ffff640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff91406bbe0 0x7ff91406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.545+0000 7ff926ffd640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff91406bbe0 0x7ff91406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.545+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7ff9180ab8d0 con 0x7ff92806e3e0 2024-02-18T11:47:20.540 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.545+0000 7ff926ffd640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff91406bbe0 0x7ff91406e0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009ac0 tx=0x7ff91c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:20.542 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.549+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9180771b0 con 0x7ff92806e3e0 2024-02-18T11:47:20.715 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.721+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9280cf110 con 0x7ff92806e3e0 2024-02-18T11:47:20.715 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.722+0000 7ff90ffff640 1 -- 172.21.15.116:0/3279120699 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7ff91807a870 con 0x7ff92806e3e0 2024-02-18T11:47:20.715 INFO:teuthology.misc.health.smithi116.stdout: 2024-02-18T11:47:20.716 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":12,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:18.295410+0000","last_up_change":"0.000000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:20.717 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff91406bbe0 msgr2=0x7ff91406e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:20.717 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff91406bbe0 0x7ff91406e0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009ac0 tx=0x7ff91c008040 comp rx=0 tx=0).stop 2024-02-18T11:47:20.717 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 msgr2=0x7ff9280ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 0x7ff9280ae050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff918000c00 tx=0x7ff9180046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 shutdown_connections 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff91406bbe0 0x7ff91406e0a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff92815dd40 0x7ff92806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff9280ccbb0 0x7ff92813f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 --2- 172.21.15.116:0/3279120699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92806e3e0 0x7ff9280ae050 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 >> 172.21.15.116:0/3279120699 conn(0x7ff92806ab60 msgr2=0x7ff9280afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 shutdown_connections 2024-02-18T11:47:20.718 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:20.724+0000 7ff92e215640 1 -- 172.21.15.116:0/3279120699 wait complete. 2024-02-18T11:47:20.727 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-02-18T11:47:27.535 DEBUG:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:27.634 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.641+0000 7f7824af4640 1 Processor -- start 2024-02-18T11:47:27.634 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.641+0000 7f7824af4640 1 -- start start 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f7824af4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f782006bf80 0x7f782015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f7824af4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f782015f120 0x7f78200b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f7824af4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78200b2b50 con 0x7f78200b2770 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f7824af4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78200ce780 con 0x7f782006bf80 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f7824af4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78200cea50 con 0x7f782015f120 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f781dd9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f782006bf80 0x7f782015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f781e59c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f78200b2770 0x7f78200ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:42980/0 (socket says 172.21.15.116:42980) 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.642+0000 7f781e59c640 1 -- 172.21.15.116:0/3740229908 learned_addr learned my addr 172.21.15.116:0/3740229908 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:27.636 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781ed9d640 1 --2- 172.21.15.116:0/3740229908 >> v2:172.21.15.143:3300/0 conn(0x7f782015f120 0x7f78200b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781dd9b640 1 -- 172.21.15.116:0/3740229908 >> v2:172.21.15.143:3300/0 conn(0x7f782015f120 msgr2=0x7f78200b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781dd9b640 1 --2- 172.21.15.116:0/3740229908 >> v2:172.21.15.143:3300/0 conn(0x7f782015f120 0x7f78200b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781dd9b640 1 -- 172.21.15.116:0/3740229908 >> v1:172.21.15.116:6789/0 conn(0x7f78200b2770 legacy=0x7f78200ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781d59a640 1 -- 172.21.15.116:0/3740229908 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169545788 0 0) 0x7f78200b2b50 con 0x7f78200b2770 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781dd9b640 1 -- 172.21.15.116:0/3740229908 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200ced30 con 0x7f782006bf80 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.643+0000 7f781dd9b640 1 --2- 172.21.15.116:0/3740229908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006bf80 0x7f782015ebe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7820074fc0 tx=0x7f780802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531f9505b0643ff3 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f781d59a640 1 -- 172.21.15.116:0/3740229908 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780802e070 con 0x7f782006bf80 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f781d59a640 1 -- 172.21.15.116:0/3740229908 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78080315a0 con 0x7f782006bf80 2024-02-18T11:47:27.637 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f781d59a640 1 -- 172.21.15.116:0/3740229908 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808031930 con 0x7f782006bf80 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 -- 172.21.15.116:0/3740229908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006bf80 msgr2=0x7f782015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 --2- 172.21.15.116:0/3740229908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006bf80 0x7f782015ebe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7820074fc0 tx=0x7f780802b840 comp rx=0 tx=0).stop 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 -- 172.21.15.116:0/3740229908 shutdown_connections 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 --2- 172.21.15.116:0/3740229908 >> v2:172.21.15.143:3300/0 conn(0x7f782015f120 0x7f78200b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 --2- 172.21.15.116:0/3740229908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782006bf80 0x7f782015ebe0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 -- 172.21.15.116:0/3740229908 >> 172.21.15.116:0/3740229908 conn(0x7f782008af90 msgr2=0x7f782008b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 -- 172.21.15.116:0/3740229908 shutdown_connections 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.644+0000 7f7824af4640 1 -- 172.21.15.116:0/3740229908 wait complete. 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 Processor -- start 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 -- start start 2024-02-18T11:47:27.640 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f782006bf80 0x7f782014f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.641 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 0x7f78200b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.641 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782015f120 0x7f782014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.641 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200b6390 con 0x7f78200b2770 2024-02-18T11:47:27.641 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78200b5780 con 0x7f782015f120 2024-02-18T11:47:27.641 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.646+0000 7f7824af4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78200b5a60 con 0x7f782006bf80 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 0x7f78200b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 0x7f78200b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51616/0 (socket says 172.21.15.116:51616) 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 -- 172.21.15.116:0/3544584166 learned_addr learned my addr 172.21.15.116:0/3544584166 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 -- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f782006bf80 msgr2=0x7f782014f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f782006bf80 0x7f782014f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781ed9d640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782015f120 0x7f782014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 -- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782015f120 msgr2=0x7f782014fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782015f120 0x7f782014fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 -- 172.21.15.116:0/3544584166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808009400 con 0x7f78200b2770 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f781dd9b640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 0x7f78200b6820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7808000c00 tx=0x7f78080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808004230 con 0x7f78200b2770 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7808004390 con 0x7f78200b2770 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78200b5d40 con 0x7f78200b2770 2024-02-18T11:47:27.642 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808032880 con 0x7f78200b2770 2024-02-18T11:47:27.643 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.647+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78200b5410 con 0x7f78200b2770 2024-02-18T11:47:27.643 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.648+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78200680c0 con 0x7f78200b2770 2024-02-18T11:47:27.643 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.649+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f78080329e0 con 0x7f78200b2770 2024-02-18T11:47:27.644 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.649+0000 7f7816ffd640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f780006bc70 0x7f780006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.644 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.649+0000 7f781e59c640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f780006bc70 0x7f780006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.644 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.649+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f78080ac510 con 0x7f78200b2770 2024-02-18T11:47:27.644 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.649+0000 7f781e59c640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f780006bc70 0x7f780006e130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7810001560 tx=0x7f7810007ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:27.646 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.654+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7808077160 con 0x7f78200b2770 2024-02-18T11:47:27.817 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.824+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78200b64d0 con 0x7f78200b2770 2024-02-18T11:47:27.817 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.825+0000 7f7816ffd640 1 -- 172.21.15.116:0/3544584166 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7f78200b64d0 con 0x7f78200b2770 2024-02-18T11:47:27.818 INFO:teuthology.misc.health.smithi116.stdout: 2024-02-18T11:47:27.818 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":16,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:24.481489+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:27.819 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f780006bc70 msgr2=0x7f780006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f780006bc70 0x7f780006e130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7810001560 tx=0x7f7810007ca0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 msgr2=0x7f78200b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 0x7f78200b6820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7808000c00 tx=0x7f78080046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 shutdown_connections 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f780006bc70 0x7f780006e130 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f782015f120 0x7f782014fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78200b2770 0x7f78200b6820 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 --2- 172.21.15.116:0/3544584166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f782006bf80 0x7f782014f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 >> 172.21.15.116:0/3544584166 conn(0x7f782008af90 msgr2=0x7f78200ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 shutdown_connections 2024-02-18T11:47:27.820 INFO:teuthology.misc.health.smithi116.stderr:2024-02-18T11:47:27.827+0000 7f7824af4640 1 -- 172.21.15.116:0/3544584166 wait complete. 2024-02-18T11:47:27.829 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-02-18T11:47:27.829 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-02-18T11:47:27.829 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-18T11:47:27.830 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-18T11:47:27.832 INFO:tasks.ceph.mds.a:Started 2024-02-18T11:47:27.832 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-18T11:47:27.832 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-18T11:47:27.834 INFO:tasks.ceph.mds.c:Started 2024-02-18T11:47:27.834 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-18T11:47:27.834 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-18T11:47:27.837 INFO:tasks.ceph.mds.b:Started 2024-02-18T11:47:27.837 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-18T11:47:27.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-18T11:47:27.839 INFO:tasks.ceph.mds.d:Started 2024-02-18T11:47:27.839 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-02-18T11:47:27.841 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:27.907 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-02-18T11:47:27.907 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-02-18T11:47:27.907 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T11:47:27.915+0000 7fc5bb3bb640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-02-18T11:47:27.922 INFO:tasks.ceph.mds.d.smithi143.stdout:starting mds.d at 2024-02-18T11:47:27.922 INFO:tasks.ceph.mds.b.smithi143.stdout:starting mds.b at 2024-02-18T11:47:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.953+0000 7fcfa11b6640 1 Processor -- start 2024-02-18T11:47:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.953+0000 7fcfa11b6640 1 -- start start 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcfa11b6640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcf9c15dc60 0x7fcf9c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcfa11b6640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 0x7fcf9c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcfa11b6640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9c0ce780 con 0x7fcf9c161850 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcfa11b6640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c0d3730 con 0x7fcf9c15dc60 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcfa11b6640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c0cf250 con 0x7fcf9c0d1080 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 0x7fcf9c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 0x7fcf9c0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:41230/0 (socket says 172.21.15.116:41230) 2024-02-18T11:47:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcf9b59e640 1 -- 172.21.15.116:0/1087419234 learned_addr learned my addr 172.21.15.116:0/1087419234 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.954+0000 7fcf9ad9d640 1 --2- 172.21.15.116:0/1087419234 >> v2:172.21.15.143:3300/0 conn(0x7fcf9c15dc60 0x7fcf9c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.955+0000 7fcf9b59e640 1 -- 172.21.15.116:0/1087419234 >> v2:172.21.15.143:3300/0 conn(0x7fcf9c15dc60 msgr2=0x7fcf9c0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.955+0000 7fcf9b59e640 1 --2- 172.21.15.116:0/1087419234 >> v2:172.21.15.143:3300/0 conn(0x7fcf9c15dc60 0x7fcf9c0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.955+0000 7fcf9b59e640 1 -- 172.21.15.116:0/1087419234 >> v1:172.21.15.143:6789/0 conn(0x7fcf9c161850 legacy=0x7fcf9c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.955+0000 7fcf9b59e640 1 -- 172.21.15.116:0/1087419234 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c0ce9b0 con 0x7fcf9c0d1080 2024-02-18T11:47:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.955+0000 7fcf9b59e640 1 --2- 172.21.15.116:0/1087419234 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 0x7fcf9c0ce240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009e50 tx=0x7fcf9402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ea7c252751dd2f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcf99d9b640 1 -- 172.21.15.116:0/1087419234 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9402cb00 con 0x7fcf9c0d1080 2024-02-18T11:47:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcf99d9b640 1 -- 172.21.15.116:0/1087419234 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9402cc60 con 0x7fcf9c0d1080 2024-02-18T11:47:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcf99d9b640 1 -- 172.21.15.116:0/1087419234 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940343f0 con 0x7fcf9c0d1080 2024-02-18T11:47:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 -- 172.21.15.116:0/1087419234 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 msgr2=0x7fcf9c0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/1087419234 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 0x7fcf9c0ce240 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009e50 tx=0x7fcf9402bf60 comp rx=0 tx=0).stop 2024-02-18T11:47:27.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 -- 172.21.15.116:0/1087419234 shutdown_connections 2024-02-18T11:47:27.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/1087419234 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c0d1080 0x7fcf9c0ce240 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/1087419234 >> v2:172.21.15.143:3300/0 conn(0x7fcf9c15dc60 0x7fcf9c0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 -- 172.21.15.116:0/1087419234 >> 172.21.15.116:0/1087419234 conn(0x7fcf9c0b5910 msgr2=0x7fcf9c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:27.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 -- 172.21.15.116:0/1087419234 shutdown_connections 2024-02-18T11:47:27.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.956+0000 7fcfa11b6640 1 -- 172.21.15.116:0/1087419234 wait complete. 2024-02-18T11:47:27.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 Processor -- start 2024-02-18T11:47:27.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 -- start start 2024-02-18T11:47:27.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf9c0d1080 0x7fcf9c15fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c15dc60 0x7fcf9c1437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 0x7fcf9c06c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c160e70 con 0x7fcf9c161850 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c151180 con 0x7fcf9c0d1080 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcfa11b6640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c143ce0 con 0x7fcf9c15dc60 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 0x7fcf9c06c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 0x7fcf9c06c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51648/0 (socket says 172.21.15.116:51648) 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcf9b59e640 1 -- 172.21.15.116:0/239644175 learned_addr learned my addr 172.21.15.116:0/239644175 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcf9a59c640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c15dc60 0x7fcf9c1437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.958+0000 7fcf9ad9d640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf9c0d1080 0x7fcf9c15fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf9b59e640 1 -- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c15dc60 msgr2=0x7fcf9c1437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf9b59e640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c15dc60 0x7fcf9c1437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf9b59e640 1 -- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf9c0d1080 msgr2=0x7fcf9c15fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:27.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf9b59e640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf9c0d1080 0x7fcf9c15fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf9b59e640 1 -- 172.21.15.116:0/239644175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf94009b30 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf9b59e640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 0x7fcf9c06c930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf94036040 tx=0x7fcf94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94009510 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9403e450 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c1601f0 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9403e800 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.959+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c1607b0 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.960+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c067840 con 0x7fcf9c161850 2024-02-18T11:47:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.960+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fcf9403e960 con 0x7fcf9c161850 2024-02-18T11:47:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.961+0000 7fcf837fe640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fcf8806bc70 0x7fcf8806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.961+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcf940ad070 con 0x7fcf9c161850 2024-02-18T11:47:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.961+0000 7fcf9ad9d640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fcf8806bc70 0x7fcf8806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.961+0000 7fcf9ad9d640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fcf8806bc70 0x7fcf8806e130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009b40 tx=0x7fcf8c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:27.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:27.965+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf94077c50 con 0x7fcf9c161850 2024-02-18T11:47:28.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.137+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf9c160fb0 con 0x7fcf9c161850 2024-02-18T11:47:28.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.137+0000 7fcf837fe640 1 -- 172.21.15.116:0/239644175 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7fcf9c160fb0 con 0x7fcf9c161850 2024-02-18T11:47:28.130 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:28.130 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":16,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:24.481489+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fcf8806bc70 msgr2=0x7fcf8806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fcf8806bc70 0x7fcf8806e130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009b40 tx=0x7fcf8c009190 comp rx=0 tx=0).stop 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 msgr2=0x7fcf9c06c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 0x7fcf9c06c930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf94036040 tx=0x7fcf94004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 shutdown_connections 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fcf8806bc70 0x7fcf8806e130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf9c161850 0x7fcf9c06c930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcf9c15dc60 0x7fcf9c1437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 --2- 172.21.15.116:0/239644175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf9c0d1080 0x7fcf9c15fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.139+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 >> 172.21.15.116:0/239644175 conn(0x7fcf9c0b5910 msgr2=0x7fcf9c073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:28.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.140+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 shutdown_connections 2024-02-18T11:47:28.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.140+0000 7fcfa11b6640 1 -- 172.21.15.116:0/239644175 wait complete. 2024-02-18T11:47:28.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:47:28.143 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:47:28.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f16113eb640 1 Processor -- start 2024-02-18T11:47:28.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f16113eb640 1 -- start start 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f16113eb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f160c1617e0 0x7f160c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f16113eb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0cf0d0 con 0x7f160c15dbf0 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f16113eb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f160c0d1ee0 con 0x7f160c0cec80 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f16113eb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f160c161dc0 con 0x7f160c1617e0 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.254+0000 7f160a7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f160c1617e0 0x7f160c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f160affd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f160c15dbf0 0x7f160c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:42996/0 (socket says 172.21.15.116:42996) 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f160affd640 1 -- 172.21.15.116:0/2093645924 learned_addr learned my addr 172.21.15.116:0/2093645924 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f160a7fc640 1 -- 172.21.15.116:0/2093645924 >> v1:172.21.15.143:6789/0 conn(0x7f160c0cec80 legacy=0x7f160c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f160a7fc640 1 -- 172.21.15.116:0/2093645924 >> v1:172.21.15.116:6789/0 conn(0x7f160c15dbf0 legacy=0x7f160c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f160a7fc640 1 -- 172.21.15.116:0/2093645924 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c1620a0 con 0x7f160c1617e0 2024-02-18T11:47:28.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f1609ffb640 1 -- 172.21.15.116:0/2093645924 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456800700 0 0) 0x7f15fc001490 con 0x7f160c15dbf0 2024-02-18T11:47:28.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.255+0000 7f160a7fc640 1 --2- 172.21.15.116:0/2093645924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c1617e0 0x7f160c0ce740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16000097a0 tx=0x7f160002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3828350ccec5b9a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:28.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f1609ffb640 1 -- 172.21.15.116:0/2093645924 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160002e070 con 0x7f160c1617e0 2024-02-18T11:47:28.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f1609ffb640 1 -- 172.21.15.116:0/2093645924 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16000315a0 con 0x7f160c1617e0 2024-02-18T11:47:28.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f1609ffb640 1 -- 172.21.15.116:0/2093645924 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600031930 con 0x7f160c1617e0 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 -- 172.21.15.116:0/2093645924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c1617e0 msgr2=0x7f160c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 --2- 172.21.15.116:0/2093645924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c1617e0 0x7f160c0ce740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16000097a0 tx=0x7f160002b840 comp rx=0 tx=0).stop 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 -- 172.21.15.116:0/2093645924 shutdown_connections 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 --2- 172.21.15.116:0/2093645924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c1617e0 0x7f160c0ce740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 -- 172.21.15.116:0/2093645924 >> 172.21.15.116:0/2093645924 conn(0x7f160c0b5480 msgr2=0x7f160c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 -- 172.21.15.116:0/2093645924 shutdown_connections 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.256+0000 7f16113eb640 1 -- 172.21.15.116:0/2093645924 wait complete. 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.257+0000 7f16113eb640 1 Processor -- start 2024-02-18T11:47:28.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 -- start start 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f160c0cec80 0x7f160c147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 0x7f160c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f160c1617e0 0x7f160c156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c1554e0 con 0x7f160c15dbf0 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c1548d0 con 0x7f160c1617e0 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f16113eb640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160c154bb0 con 0x7f160c0cec80 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f160a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 0x7f160c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f160a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 0x7f160c153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51668/0 (socket says 172.21.15.116:51668) 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.258+0000 7f160a7fc640 1 -- 172.21.15.116:0/3124862318 learned_addr learned my addr 172.21.15.116:0/3124862318 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160b7fe640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f160c1617e0 0x7f160c156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160affd640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f160c0cec80 0x7f160c147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160a7fc640 1 -- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f160c0cec80 msgr2=0x7f160c147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160a7fc640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f160c0cec80 0x7f160c147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160a7fc640 1 -- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f160c1617e0 msgr2=0x7f160c156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160a7fc640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f160c1617e0 0x7f160c156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160a7fc640 1 -- 172.21.15.116:0/3124862318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009400 con 0x7f160c15dbf0 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160b7fe640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f160c1617e0 0x7f160c156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160affd640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f160c0cec80 0x7f160c147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f160a7fc640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 0x7f160c153ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1600034040 tx=0x7f16000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600032900 con 0x7f160c15dbf0 2024-02-18T11:47:28.254 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600032a60 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c15e150 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600032e10 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.259+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c155170 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.260+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c067840 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.260+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f160003b3f0 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.261+0000 7f15f37fe640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f806bca0 0x7f15f806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.261+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f16000ac4a0 con 0x7f160c15dbf0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.261+0000 7f160affd640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f806bca0 0x7f15f806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.261+0000 7f160affd640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f806bca0 0x7f15f806e160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0016f0 tx=0x7f15fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:28.257 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.265+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1600076fd0 con 0x7f160c15dbf0 2024-02-18T11:47:28.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.437+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f160c074520 con 0x7f160c15dbf0 2024-02-18T11:47:28.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.437+0000 7f15f37fe640 1 -- 172.21.15.116:0/3124862318 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f160007a690 con 0x7f160c15dbf0 2024-02-18T11:47:28.430 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:47:28.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f806bca0 msgr2=0x7f15f806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f806bca0 0x7f15f806e160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0016f0 tx=0x7f15fc009040 comp rx=0 tx=0).stop 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 msgr2=0x7f160c153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 0x7f160c153ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1600034040 tx=0x7f16000046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 shutdown_connections 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f806bca0 0x7f15f806e160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f160c1617e0 0x7f160c156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160c15dbf0 0x7f160c153ef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 --2- 172.21.15.116:0/3124862318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f160c0cec80 0x7f160c147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 >> 172.21.15.116:0/3124862318 conn(0x7f160c0b5480 msgr2=0x7f160c0d3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 shutdown_connections 2024-02-18T11:47:28.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.439+0000 7f16113eb640 1 -- 172.21.15.116:0/3124862318 wait complete. 2024-02-18T11:47:28.442 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:47:28.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.550+0000 7f369f10d640 1 Processor -- start 2024-02-18T11:47:28.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.550+0000 7f369f10d640 1 -- start start 2024-02-18T11:47:28.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f369f10d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f369815dc60 0x7f36980b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f369f10d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3698161850 0x7f36980ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f369f10d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36980ce780 con 0x7f36980d0970 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f369f10d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36980d3730 con 0x7f3698161850 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f369f10d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980cf250 con 0x7f369815dc60 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f3697fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3698161850 0x7f36980ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f3697fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3698161850 0x7f36980ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51684/0 (socket says 172.21.15.116:51684) 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.551+0000 7f3697fff640 1 -- 172.21.15.116:0/2009114358 learned_addr learned my addr 172.21.15.116:0/2009114358 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f369cea9640 1 --2- 172.21.15.116:0/2009114358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f369815dc60 0x7f36980b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f3697fff640 1 -- 172.21.15.116:0/2009114358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f369815dc60 msgr2=0x7f36980b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f3697fff640 1 --2- 172.21.15.116:0/2009114358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f369815dc60 0x7f36980b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f3697fff640 1 -- 172.21.15.116:0/2009114358 >> v1:172.21.15.143:6789/0 conn(0x7f36980d0970 legacy=0x7f36980d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f3697fff640 1 -- 172.21.15.116:0/2009114358 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980ce9b0 con 0x7f3698161850 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f3697fff640 1 --2- 172.21.15.116:0/2009114358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36980ce240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa1067e6adff458 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f36977fe640 1 -- 172.21.15.116:0/2009114358 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c031640 con 0x7f3698161850 2024-02-18T11:47:28.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f36977fe640 1 -- 172.21.15.116:0/2009114358 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c0317a0 con 0x7f3698161850 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.552+0000 7f36977fe640 1 -- 172.21.15.116:0/2009114358 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c031b30 con 0x7f3698161850 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 -- 172.21.15.116:0/2009114358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 msgr2=0x7f36980ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 --2- 172.21.15.116:0/2009114358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36980ce240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02b8f0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 -- 172.21.15.116:0/2009114358 shutdown_connections 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 --2- 172.21.15.116:0/2009114358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36980ce240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 --2- 172.21.15.116:0/2009114358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f369815dc60 0x7f36980b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 -- 172.21.15.116:0/2009114358 >> 172.21.15.116:0/2009114358 conn(0x7f36980b5910 msgr2=0x7f36980b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 -- 172.21.15.116:0/2009114358 shutdown_connections 2024-02-18T11:47:28.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.553+0000 7f369f10d640 1 -- 172.21.15.116:0/2009114358 wait complete. 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.554+0000 7f369f10d640 1 Processor -- start 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.554+0000 7f369f10d640 1 -- start start 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369f10d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36980d0970 0x7f36980c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369f10d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f369815dc60 0x7f36980cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369f10d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36981560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369f10d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980cade0 con 0x7f3698161850 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369f10d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980caab0 con 0x7f369815dc60 2024-02-18T11:47:28.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369f10d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36980cd8d0 con 0x7f36980d0970 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36981560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36981560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51694/0 (socket says 172.21.15.116:51694) 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 -- 172.21.15.116:0/1007551022 learned_addr learned my addr 172.21.15.116:0/1007551022 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f3697fff640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f369815dc60 0x7f36980cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369cea9640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36980d0970 0x7f36980c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 -- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36980d0970 msgr2=0x7f36980c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36980d0970 0x7f36980c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 -- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f369815dc60 msgr2=0x7f36980cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f369815dc60 0x7f36980cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369d6aa640 1 -- 172.21.15.116:0/1007551022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0094d0 con 0x7f3698161850 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.555+0000 7f369cea9640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36980d0970 0x7f36980c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.556+0000 7f369d6aa640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36981560b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f369000c780 tx=0x7f369000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.556+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690004b60 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.556+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369000ce90 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.556+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36981531e0 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.556+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000a850 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.556+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36981537a0 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.557+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3698067840 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.557+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f369000a9b0 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.557+0000 7f36957fa640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f368406bca0 0x7f368406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.558+0000 7f369cea9640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f368406bca0 0x7f368406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.558+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f369008a820 con 0x7f3698161850 2024-02-18T11:47:28.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.558+0000 7f369cea9640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f368406bca0 0x7f368406e160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f368800a9d0 tx=0x7f3688008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.562+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3690055350 con 0x7f3698161850 2024-02-18T11:47:28.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.769+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3698074520 con 0x7f3698161850 2024-02-18T11:47:28.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.769+0000 7f36957fa640 1 -- 172.21.15.116:0/1007551022 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f3690058a10 con 0x7f3698161850 2024-02-18T11:47:28.762 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:28.762 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-18T11:47:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 2 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f368406bca0 msgr2=0x7f368406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f368406bca0 0x7f368406e160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f368800a9d0 tx=0x7f3688008040 comp rx=0 tx=0).stop 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 msgr2=0x7f36981560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36981560b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f369000c780 tx=0x7f369000cc40 comp rx=0 tx=0).stop 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 shutdown_connections 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f368406bca0 0x7f368406e160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3698161850 0x7f36981560b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f369815dc60 0x7f36980cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 --2- 172.21.15.116:0/1007551022 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36980d0970 0x7f36980c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 >> 172.21.15.116:0/1007551022 conn(0x7f36980b5910 msgr2=0x7f36980cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 shutdown_connections 2024-02-18T11:47:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:28.771+0000 7f369f10d640 1 -- 172.21.15.116:0/1007551022 wait complete. 2024-02-18T11:47:30.775 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:47:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.896+0000 7fc10bd8a640 1 Processor -- start 2024-02-18T11:47:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.896+0000 7fc10bd8a640 1 -- start start 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.896+0000 7fc10bd8a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc104160e90 0x7fc1040afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc10bd8a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc104070890 con 0x7fc10406aff0 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc10bd8a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10406b480 con 0x7fc10406ab60 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc10bd8a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1040d14a0 con 0x7fc104160e90 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc109b26640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc104160e90 0x7fc1040afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc10a327640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc10406aff0 0x7fc104070180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:42998/0 (socket says 172.21.15.116:42998) 2024-02-18T11:47:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc10a327640 1 -- 172.21.15.116:0/3701032594 learned_addr learned my addr 172.21.15.116:0/3701032594 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc109b26640 1 -- 172.21.15.116:0/3701032594 >> v1:172.21.15.143:6789/0 conn(0x7fc10406ab60 legacy=0x7fc10406fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc109b26640 1 -- 172.21.15.116:0/3701032594 >> v1:172.21.15.116:6789/0 conn(0x7fc10406aff0 legacy=0x7fc104070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.897+0000 7fc109b26640 1 -- 172.21.15.116:0/3701032594 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040d0400 con 0x7fc104160e90 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc109b26640 1 --2- 172.21.15.116:0/3701032594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc104160e90 0x7fc1040afac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc100009dc0 tx=0x7fc10002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c06f764caca467 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc108b24640 1 -- 172.21.15.116:0/3701032594 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10002cc40 con 0x7fc104160e90 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc108b24640 1 -- 172.21.15.116:0/3701032594 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10002cda0 con 0x7fc104160e90 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc108b24640 1 -- 172.21.15.116:0/3701032594 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100034660 con 0x7fc104160e90 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc10bd8a640 1 -- 172.21.15.116:0/3701032594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc104160e90 msgr2=0x7fc1040afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/3701032594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc104160e90 0x7fc1040afac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc100009dc0 tx=0x7fc10002be90 comp rx=0 tx=0).stop 2024-02-18T11:47:30.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc10bd8a640 1 -- 172.21.15.116:0/3701032594 shutdown_connections 2024-02-18T11:47:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/3701032594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc104160e90 0x7fc1040afac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.898+0000 7fc10bd8a640 1 -- 172.21.15.116:0/3701032594 >> 172.21.15.116:0/3701032594 conn(0x7fc1040b23f0 msgr2=0x7fc1040b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.899+0000 7fc10bd8a640 1 -- 172.21.15.116:0/3701032594 shutdown_connections 2024-02-18T11:47:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.899+0000 7fc10bd8a640 1 -- 172.21.15.116:0/3701032594 wait complete. 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 Processor -- start 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 -- start start 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc10406ab60 0x7fc104150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 0x7fc1041f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc104160e90 0x7fc10406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041f7a10 con 0x7fc10406aff0 2024-02-18T11:47:30.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc104151d50 con 0x7fc104160e90 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc10bd8a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc104152810 con 0x7fc10406ab60 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc109325640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 0x7fc1041f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc109325640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 0x7fc1041f74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51716/0 (socket says 172.21.15.116:51716) 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.900+0000 7fc109325640 1 -- 172.21.15.116:0/2860514911 learned_addr learned my addr 172.21.15.116:0/2860514911 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109b26640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc10406ab60 0x7fc104150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc10a327640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc104160e90 0x7fc10406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109325640 1 -- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc10406ab60 msgr2=0x7fc104150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109325640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc10406ab60 0x7fc104150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109325640 1 -- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc104160e90 msgr2=0x7fc10406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109325640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc104160e90 0x7fc10406e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109325640 1 -- 172.21.15.116:0/2860514911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100009a70 con 0x7fc10406aff0 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc109325640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 0x7fc1041f74d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c410 tx=0x7fc0f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40035f0 con 0x7fc10406aff0 2024-02-18T11:47:30.895 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4003750 con 0x7fc10406aff0 2024-02-18T11:47:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc104152af0 con 0x7fc10406aff0 2024-02-18T11:47:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4003b40 con 0x7fc10406aff0 2024-02-18T11:47:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.901+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1041522d0 con 0x7fc10406aff0 2024-02-18T11:47:30.896 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.902+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc104152d10 con 0x7fc10406aff0 2024-02-18T11:47:30.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.902+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc0f4003ca0 con 0x7fc10406aff0 2024-02-18T11:47:30.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.903+0000 7fc0fa7fc640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc0e806bc70 0x7fc0e806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:30.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.903+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc0f4089b10 con 0x7fc10406aff0 2024-02-18T11:47:30.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.903+0000 7fc109b26640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc0e806bc70 0x7fc0e806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:30.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.903+0000 7fc109b26640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc0e806bc70 0x7fc0e806e130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc100036000 tx=0x7fc100030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:30.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:30.907+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc104152d10 con 0x7fc10406aff0 2024-02-18T11:47:31.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.112+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc104152ce0 con 0x7fc10406aff0 2024-02-18T11:47:31.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.112+0000 7fc0fa7fc640 1 -- 172.21.15.116:0/2860514911 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7fc0f4017020 con 0x7fc10406aff0 2024-02-18T11:47:31.105 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:31.105 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-02-18T11:47:31.105 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 3 2024-02-18T11:47:31.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc0e806bc70 msgr2=0x7fc0e806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc0e806bc70 0x7fc0e806e130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc100036000 tx=0x7fc100030000 comp rx=0 tx=0).stop 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 msgr2=0x7fc1041f74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 0x7fc1041f74d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c410 tx=0x7fc0f400c8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 shutdown_connections 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc0e806bc70 0x7fc0e806e130 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc104160e90 0x7fc10406e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc10406aff0 0x7fc1041f74d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 --2- 172.21.15.116:0/2860514911 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc10406ab60 0x7fc104150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 >> 172.21.15.116:0/2860514911 conn(0x7fc1040b23f0 msgr2=0x7fc1040cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 shutdown_connections 2024-02-18T11:47:31.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.114+0000 7fc10bd8a640 1 -- 172.21.15.116:0/2860514911 wait complete. 2024-02-18T11:47:31.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.225+0000 7fc478a55640 1 Processor -- start 2024-02-18T11:47:31.217 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.225+0000 7fc478a55640 1 -- start start 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc478a55640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc47406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc478a55640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4740d1fa0 con 0x7fc4741611e0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc478a55640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4740d2560 con 0x7fc47406c5b0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc478a55640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47406ca40 con 0x7fc4740b6af0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc47259c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc4741611e0 0x7fc4740cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:43000/0 (socket says 172.21.15.116:43000) 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc47259c640 1 -- 172.21.15.116:0/213780054 learned_addr learned my addr 172.21.15.116:0/213780054 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc469d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc47406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962174224 0 0) 0x7fc4740d1fa0 con 0x7fc4741611e0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc460003520 con 0x7fc4741611e0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135350898 0 0) 0x7fc4740d2560 con 0x7fc47406c5b0 2024-02-18T11:47:31.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4740d1fa0 con 0x7fc47406c5b0 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1119648471 0 0) 0x7fc460003520 con 0x7fc4741611e0 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 msgr2=0x7fc47406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc47406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 >> v1:172.21.15.143:6789/0 conn(0x7fc47406c5b0 legacy=0x7fc4740d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4740cf110 con 0x7fc4741611e0 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fc464001670 con 0x7fc4741611e0 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc469d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc47406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 >> v1:172.21.15.116:6789/0 conn(0x7fc4741611e0 legacy=0x7fc4740cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460003b30 0x7fc460003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4600047c0 0x7fc460004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc460005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47406ca40 con 0x7fc4600047c0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46000b9b0 con 0x7fc460003b30 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46000bb70 con 0x7fc4740b6af0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc469d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4600047c0 0x7fc460004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc47259c640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460003b30 0x7fc460003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.226+0000 7fc472d9d640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc460005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc469d9b640 1 -- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 msgr2=0x7fc460005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc469d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc460005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc469d9b640 1 -- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460003b30 msgr2=0x7fc460003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc469d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc460003b30 0x7fc460003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc469d9b640 1 -- 172.21.15.116:0/213780054 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c009400 con 0x7fc4600047c0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc472d9d640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc4740b6af0 0x7fc460005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc469d9b640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4600047c0 0x7fc460004c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009c60 tx=0x7fc4600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29f8afb864047bc server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c02c8a0 con 0x7fc4600047c0 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c02ca00 con 0x7fc4600047c0 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.227+0000 7fc471d9b640 1 -- 172.21.15.116:0/213780054 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c02cdb0 con 0x7fc4600047c0 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc478a55640 1 -- 172.21.15.116:0/213780054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4600047c0 msgr2=0x7fc460004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc478a55640 1 --2- 172.21.15.116:0/213780054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4600047c0 0x7fc460004c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009c60 tx=0x7fc4600057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc472d9d640 1 -- 172.21.15.116:0/213780054 reap_dead start 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc478a55640 1 -- 172.21.15.116:0/213780054 shutdown_connections 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc478a55640 1 -- 172.21.15.116:0/213780054 >> 172.21.15.116:0/213780054 conn(0x7fc4740b5b00 msgr2=0x7fc4740d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc478a55640 1 -- 172.21.15.116:0/213780054 shutdown_connections 2024-02-18T11:47:31.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.228+0000 7fc478a55640 1 -- 172.21.15.116:0/213780054 wait complete. 2024-02-18T11:47:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.229+0000 7fc478a55640 1 Processor -- start 2024-02-18T11:47:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.229+0000 7fc478a55640 1 -- start start 2024-02-18T11:47:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc478a55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4600047c0 0x7fc4740cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc478a55640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 0x7fc4741591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc478a55640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc47415dca0 0x7fc47415f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc478a55640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47415d3f0 con 0x7fc47415d840 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc478a55640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc474159710 con 0x7fc4600047c0 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc478a55640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47415e1a0 con 0x7fc47415dca0 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 0x7fc4741591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 0x7fc4741591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51728/0 (socket says 172.21.15.116:51728) 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 -- 172.21.15.116:0/2404254338 learned_addr learned my addr 172.21.15.116:0/2404254338 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc472d9d640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc47415dca0 0x7fc47415f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc47259c640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4600047c0 0x7fc4740cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 -- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc47415dca0 msgr2=0x7fc47415f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc47415dca0 0x7fc47415f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 -- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4600047c0 msgr2=0x7fc4740cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4600047c0 0x7fc4740cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc469d9b640 1 -- 172.21.15.116:0/2404254338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474159b70 con 0x7fc47415d840 2024-02-18T11:47:31.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc472d9d640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc47415dca0 0x7fc47415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.230+0000 7fc47259c640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4600047c0 0x7fc4740cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.231+0000 7fc469d9b640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 0x7fc4741591d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc45c037040 tx=0x7fc45c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.231+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c006e30 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.231+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c034a80 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.231+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc474160400 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.231+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c034df0 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.231+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4741609c0 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.232+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc474067840 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.232+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc45c040450 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.233+0000 7fc46b7fe640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc45406bc70 0x7fc45406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.233+0000 7fc47259c640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc45406bc70 0x7fc45406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.233+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc45c0ac740 con 0x7fc47415d840 2024-02-18T11:47:31.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.233+0000 7fc47259c640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc45406bc70 0x7fc45406e130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc464001eb0 tx=0x7fc464009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.237+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc45c077300 con 0x7fc47415d840 2024-02-18T11:47:31.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.410+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc474074520 con 0x7fc47415d840 2024-02-18T11:47:31.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.411+0000 7fc46b7fe640 1 -- 172.21.15.116:0/2404254338 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12896 (secure 0 0 0) 0x7fc45c07a9c0 con 0x7fc47415d840 2024-02-18T11:47:31.404 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:31.405 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":16,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:24.481489+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:31.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc45406bc70 msgr2=0x7fc45406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc45406bc70 0x7fc45406e130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc464001eb0 tx=0x7fc464009040 comp rx=0 tx=0).stop 2024-02-18T11:47:31.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 msgr2=0x7fc4741591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 0x7fc4741591d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc45c037040 tx=0x7fc45c004730 comp rx=0 tx=0).stop 2024-02-18T11:47:31.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 shutdown_connections 2024-02-18T11:47:31.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc45406bc70 0x7fc45406e130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc47415dca0 0x7fc47415f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc47415d840 0x7fc4741591d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 --2- 172.21.15.116:0/2404254338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4600047c0 0x7fc4740cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 >> 172.21.15.116:0/2404254338 conn(0x7fc4740b5b00 msgr2=0x7fc4740d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:31.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 shutdown_connections 2024-02-18T11:47:31.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.413+0000 7fc478a55640 1 -- 172.21.15.116:0/2404254338 wait complete. 2024-02-18T11:47:31.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:47:31.417 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:47:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.528+0000 7f341d06f640 1 Processor -- start 2024-02-18T11:47:31.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.528+0000 7f341d06f640 1 -- start start 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341d06f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f34180b31a0 0x7f341806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341d06f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 0x7f341815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341d06f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34180b3610 con 0x7f3418144090 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341d06f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341806ece0 con 0x7f34180b31a0 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341d06f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34180b2a50 con 0x7f341806e870 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341759e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 0x7f341815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341759e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 0x7f341815e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:41334/0 (socket says 172.21.15.116:41334) 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341759e640 1 -- 172.21.15.116:0/1638295962 learned_addr learned my addr 172.21.15.116:0/1638295962 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.529+0000 7f341659c640 1 --2- 172.21.15.116:0/1638295962 >> v2:172.21.15.143:3300/0 conn(0x7f34180b31a0 0x7f341806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.530+0000 7f341759e640 1 -- 172.21.15.116:0/1638295962 >> v2:172.21.15.143:3300/0 conn(0x7f34180b31a0 msgr2=0x7f341806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:31.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.530+0000 7f341759e640 1 --2- 172.21.15.116:0/1638295962 >> v2:172.21.15.143:3300/0 conn(0x7f34180b31a0 0x7f341806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.530+0000 7f341759e640 1 -- 172.21.15.116:0/1638295962 >> v1:172.21.15.143:6789/0 conn(0x7f3418144090 legacy=0x7f34180cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.530+0000 7f341759e640 1 -- 172.21.15.116:0/1638295962 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34180cd0f0 con 0x7f341806e870 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.530+0000 7f341759e640 1 --2- 172.21.15.116:0/1638295962 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 0x7f341815e610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3410009e50 tx=0x7f341002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bab1897a50ae399a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f3415d9b640 1 -- 172.21.15.116:0/1638295962 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341002ca70 con 0x7f341806e870 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f3415d9b640 1 -- 172.21.15.116:0/1638295962 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341002cbd0 con 0x7f341806e870 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f3415d9b640 1 -- 172.21.15.116:0/1638295962 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34100343f0 con 0x7f341806e870 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 -- 172.21.15.116:0/1638295962 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 msgr2=0x7f341815e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 --2- 172.21.15.116:0/1638295962 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 0x7f341815e610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3410009e50 tx=0x7f341002bed0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 -- 172.21.15.116:0/1638295962 shutdown_connections 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 --2- 172.21.15.116:0/1638295962 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f341806e870 0x7f341815e610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 --2- 172.21.15.116:0/1638295962 >> v2:172.21.15.143:3300/0 conn(0x7f34180b31a0 0x7f341806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 -- 172.21.15.116:0/1638295962 >> 172.21.15.116:0/1638295962 conn(0x7f3418067000 msgr2=0x7f341806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:31.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 -- 172.21.15.116:0/1638295962 shutdown_connections 2024-02-18T11:47:31.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.531+0000 7f341d06f640 1 -- 172.21.15.116:0/1638295962 wait complete. 2024-02-18T11:47:31.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.533+0000 7f341d06f640 1 Processor -- start 2024-02-18T11:47:31.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.533+0000 7f341d06f640 1 -- start start 2024-02-18T11:47:31.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 0x7f341813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34180b31a0 0x7f341813d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418144090 0x7f34181f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341813eed0 con 0x7f341806e870 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34181424e0 con 0x7f3418144090 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34181427c0 con 0x7f34180b31a0 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 0x7f341813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 0x7f341813bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51734/0 (socket says 172.21.15.116:51734) 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 -- 172.21.15.116:0/1373009572 learned_addr learned my addr 172.21.15.116:0/1373009572 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341659c640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34180b31a0 0x7f341813d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341759e640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418144090 0x7f34181f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 -- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34180b31a0 msgr2=0x7f341813d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34180b31a0 0x7f341813d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 -- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418144090 msgr2=0x7f34181f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418144090 0x7f34181f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 -- 172.21.15.116:0/1373009572 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3410009b30 con 0x7f341806e870 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341659c640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34180b31a0 0x7f341813d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f3416d9d640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 0x7f341813bb60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f340800c3c0 tx=0x7f340800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080052d0 con 0x7f341806e870 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3408005450 con 0x7f341806e870 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3418142aa0 con 0x7f341806e870 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408007260 con 0x7f341806e870 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.534+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34181413a0 con 0x7f341806e870 2024-02-18T11:47:31.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.535+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34180cbe40 con 0x7f341806e870 2024-02-18T11:47:31.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.536+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f34080073c0 con 0x7f341806e870 2024-02-18T11:47:31.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.536+0000 7f33ff7fe640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f340406bc70 0x7f340406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.536+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f34080896b0 con 0x7f341806e870 2024-02-18T11:47:31.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.536+0000 7f341659c640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f340406bc70 0x7f340406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.537+0000 7f341659c640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f340406bc70 0x7f340406e130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3418143790 tx=0x7f340c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.532 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.540+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3408054320 con 0x7f341806e870 2024-02-18T11:47:31.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.710+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3418142fc0 con 0x7f341806e870 2024-02-18T11:47:31.703 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.711+0000 7f33ff7fe640 1 -- 172.21.15.116:0/1373009572 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3418142fc0 con 0x7f341806e870 2024-02-18T11:47:31.703 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:47:31.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.712+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f340406bc70 msgr2=0x7f340406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.712+0000 7f341d06f640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f340406bc70 0x7f340406e130 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3418143790 tx=0x7f340c008040 comp rx=0 tx=0).stop 2024-02-18T11:47:31.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.712+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 msgr2=0x7f341813bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.712+0000 7f341d06f640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 0x7f341813bb60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f340800c3c0 tx=0x7f340800c880 comp rx=0 tx=0).stop 2024-02-18T11:47:31.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 shutdown_connections 2024-02-18T11:47:31.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f340406bc70 0x7f340406e130 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3418144090 0x7f34181f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34180b31a0 0x7f341813d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 --2- 172.21.15.116:0/1373009572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f341806e870 0x7f341813bb60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 >> 172.21.15.116:0/1373009572 conn(0x7f3418067000 msgr2=0x7f341806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:31.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 shutdown_connections 2024-02-18T11:47:31.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.713+0000 7f341d06f640 1 -- 172.21.15.116:0/1373009572 wait complete. 2024-02-18T11:47:31.716 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-18T11:47:31.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.823+0000 7fe0e3055640 1 Processor -- start 2024-02-18T11:47:31.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e3055640 1 -- start start 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e3055640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0dc15efb0 0x7fe0dc0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e3055640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc0cf900 0x7fe0dc06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e3055640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc0d0a80 con 0x7fe0dc06cd80 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e3055640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc15f390 con 0x7fe0dc15efb0 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e3055640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cfe70 con 0x7fe0dc0cf900 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.824+0000 7fe0e0df1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0dc15efb0 0x7fe0dc0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e15f2640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe0dc06cd80 0x7fe0dc099a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:43016/0 (socket says 172.21.15.116:43016) 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e15f2640 1 -- 172.21.15.116:0/2858039192 learned_addr learned my addr 172.21.15.116:0/2858039192 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:31.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0dbfff640 1 --2- 172.21.15.116:0/2858039192 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc0cf900 0x7fe0dc06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e0df1640 1 -- 172.21.15.116:0/2858039192 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc0cf900 msgr2=0x7fe0dc06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e0df1640 1 --2- 172.21.15.116:0/2858039192 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc0cf900 0x7fe0dc06c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e0df1640 1 -- 172.21.15.116:0/2858039192 >> v1:172.21.15.116:6789/0 conn(0x7fe0dc06cd80 legacy=0x7fe0dc099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e0df1640 1 -- 172.21.15.116:0/2858039192 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0d0150 con 0x7fe0dc15efb0 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.825+0000 7fe0e0df1640 1 --2- 172.21.15.116:0/2858039192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc0d0f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00097a0 tx=0x7fe0d002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3747a9c026954f26 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0db7fe640 1 -- 172.21.15.116:0/2858039192 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d002e070 con 0x7fe0dc15efb0 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0db7fe640 1 -- 172.21.15.116:0/2858039192 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d00315a0 con 0x7fe0dc15efb0 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0db7fe640 1 -- 172.21.15.116:0/2858039192 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0031930 con 0x7fe0dc15efb0 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 -- 172.21.15.116:0/2858039192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 msgr2=0x7fe0dc0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2858039192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc0d0f10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00097a0 tx=0x7fe0d002b840 comp rx=0 tx=0).stop 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 -- 172.21.15.116:0/2858039192 shutdown_connections 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2858039192 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc0cf900 0x7fe0dc06c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2858039192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc0d0f10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 -- 172.21.15.116:0/2858039192 >> 172.21.15.116:0/2858039192 conn(0x7fe0dc0d1790 msgr2=0x7fe0dc0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:31.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 -- 172.21.15.116:0/2858039192 shutdown_connections 2024-02-18T11:47:31.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.826+0000 7fe0e3055640 1 -- 172.21.15.116:0/2858039192 wait complete. 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.827+0000 7fe0e3055640 1 Processor -- start 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.827+0000 7fe0e3055640 1 -- start start 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e3055640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc06cd80 0x7fe0dc0c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e3055640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0dc0cf900 0x7fe0dc1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e3055640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc1610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e3055640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc156160 con 0x7fe0dc15efb0 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e3055640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0b4b30 con 0x7fe0dc0cf900 2024-02-18T11:47:31.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e3055640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0b4e10 con 0x7fe0dc06cd80 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e15f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc1610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e15f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc1610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51762/0 (socket says 172.21.15.116:51762) 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e15f2640 1 -- 172.21.15.116:0/2540714971 learned_addr learned my addr 172.21.15.116:0/2540714971 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.828+0000 7fe0e15f2640 1 -- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc06cd80 msgr2=0x7fe0dc0c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e15f2640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc06cd80 0x7fe0dc0c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e15f2640 1 -- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0dc0cf900 msgr2=0x7fe0dc1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0dbfff640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0dc0cf900 0x7fe0dc1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e15f2640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0dc0cf900 0x7fe0dc1525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e15f2640 1 -- 172.21.15.116:0/2540714971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d0009400 con 0x7fe0dc15efb0 2024-02-18T11:47:31.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e15f2640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc1610b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ca30 tx=0x7fe0d400cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400abd0 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d400ad30 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc0b50f0 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4012620 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.829+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc1615f0 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.830+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc067840 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.830+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe0d4007070 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.831+0000 7fe0d97fa640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe0c006bca0 0x7fe0c006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.831+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe0d408a3b0 con 0x7fe0dc15efb0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.831+0000 7fe0e0df1640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe0c006bca0 0x7fe0c006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:31.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.831+0000 7fe0e0df1640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe0c006bca0 0x7fe0c006e160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0036000 tx=0x7fe0d0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:31.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:31.835+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0d4054e50 con 0x7fe0dc15efb0 2024-02-18T11:47:32.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.008+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe0dc156380 con 0x7fe0dc15efb0 2024-02-18T11:47:32.000 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.008+0000 7fe0d97fa640 1 -- 172.21.15.116:0/2540714971 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe0dc156380 con 0x7fe0dc15efb0 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:47:32.001 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-02-18T11:47:32.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe0c006bca0 msgr2=0x7fe0c006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe0c006bca0 0x7fe0c006e160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0036000 tx=0x7fe0d0030000 comp rx=0 tx=0).stop 2024-02-18T11:47:32.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 msgr2=0x7fe0dc1610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc1610b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ca30 tx=0x7fe0d400cef0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 shutdown_connections 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe0c006bca0 0x7fe0c006e160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0dc15efb0 0x7fe0dc1610b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0dc0cf900 0x7fe0dc1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 --2- 172.21.15.116:0/2540714971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0dc06cd80 0x7fe0dc0c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 >> 172.21.15.116:0/2540714971 conn(0x7fe0dc0d1790 msgr2=0x7fe0dc06f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 shutdown_connections 2024-02-18T11:47:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.010+0000 7fe0e3055640 1 -- 172.21.15.116:0/2540714971 wait complete. 2024-02-18T11:47:32.017 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-18T11:47:32.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-18T11:47:32.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.128+0000 7faa5f015640 1 Processor -- start 2024-02-18T11:47:32.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.128+0000 7faa5f015640 1 -- start start 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5f015640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa5815dbf0 0x7faa581617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5f015640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa580b9370 0x7faa580ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5f015640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa580cf0d0 con 0x7faa580cec80 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5f015640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa580d1ee0 con 0x7faa580b9370 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5f015640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa58161dc0 con 0x7faa5815dbf0 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa57fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa580b9370 0x7faa580ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5d5b2640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7faa580cec80 0x7faa5806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:43028/0 (socket says 172.21.15.116:43028) 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa5d5b2640 1 -- 172.21.15.116:0/4083147661 learned_addr learned my addr 172.21.15.116:0/4083147661 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa57fff640 1 -- 172.21.15.116:0/4083147661 >> v2:172.21.15.143:3300/0 conn(0x7faa5815dbf0 msgr2=0x7faa581617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa57fff640 1 --2- 172.21.15.116:0/4083147661 >> v2:172.21.15.143:3300/0 conn(0x7faa5815dbf0 0x7faa581617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa57fff640 1 -- 172.21.15.116:0/4083147661 >> v1:172.21.15.116:6789/0 conn(0x7faa580cec80 legacy=0x7faa5806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa577fe640 1 -- 172.21.15.116:0/4083147661 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704130256 0 0) 0x7faa50001490 con 0x7faa580cec80 2024-02-18T11:47:32.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.129+0000 7faa57fff640 1 -- 172.21.15.116:0/4083147661 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa581620a0 con 0x7faa580b9370 2024-02-18T11:47:32.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa57fff640 1 --2- 172.21.15.116:0/4083147661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580b9370 0x7faa580ce740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa5806d330 tx=0x7faa4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245866d8be2f2ca9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:32.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa577fe640 1 -- 172.21.15.116:0/4083147661 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44031830 con 0x7faa580b9370 2024-02-18T11:47:32.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa577fe640 1 -- 172.21.15.116:0/4083147661 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa44031990 con 0x7faa580b9370 2024-02-18T11:47:32.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa577fe640 1 -- 172.21.15.116:0/4083147661 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44031d20 con 0x7faa580b9370 2024-02-18T11:47:32.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa5f015640 1 -- 172.21.15.116:0/4083147661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580b9370 msgr2=0x7faa580ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa5f015640 1 --2- 172.21.15.116:0/4083147661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580b9370 0x7faa580ce740 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa5806d330 tx=0x7faa4402b8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.130+0000 7faa5f015640 1 -- 172.21.15.116:0/4083147661 shutdown_connections 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.131+0000 7faa5f015640 1 --2- 172.21.15.116:0/4083147661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580b9370 0x7faa580ce740 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.131+0000 7faa5f015640 1 --2- 172.21.15.116:0/4083147661 >> v2:172.21.15.143:3300/0 conn(0x7faa5815dbf0 0x7faa581617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.131+0000 7faa5f015640 1 -- 172.21.15.116:0/4083147661 >> 172.21.15.116:0/4083147661 conn(0x7faa580b5480 msgr2=0x7faa580b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.131+0000 7faa5f015640 1 -- 172.21.15.116:0/4083147661 shutdown_connections 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.131+0000 7faa5f015640 1 -- 172.21.15.116:0/4083147661 wait complete. 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 Processor -- start 2024-02-18T11:47:32.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 -- start start 2024-02-18T11:47:32.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa580b9370 0x7faa5815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 0x7faa580c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5815dbf0 0x7faa58154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58151f40 con 0x7faa580cec80 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa580cd1b0 con 0x7faa580b9370 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.132+0000 7faa5f015640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa580cd490 con 0x7faa5815dbf0 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 0x7faa580c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 0x7faa580c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51794/0 (socket says 172.21.15.116:51794) 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 -- 172.21.15.116:0/1868560222 learned_addr learned my addr 172.21.15.116:0/1868560222 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa5d5b2640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5815dbf0 0x7faa58154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa5cdb1640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa580b9370 0x7faa5815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 -- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5815dbf0 msgr2=0x7faa58154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5815dbf0 0x7faa58154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 -- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa580b9370 msgr2=0x7faa5815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa580b9370 0x7faa5815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 -- 172.21.15.116:0/1868560222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44009400 con 0x7faa580cec80 2024-02-18T11:47:32.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa5cdb1640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa580b9370 0x7faa5815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa57fff640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 0x7faa580c9340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa44009890 tx=0x7faa440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4403ba70 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4403bbf0 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5815e150 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa440453f0 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.133+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5815a660 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.134+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa58067840 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.135+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7faa44045550 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.135+0000 7faa557fa640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa3c06bc70 0x7faa3c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.135+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faa440ac730 con 0x7faa580cec80 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.135+0000 7faa5cdb1640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa3c06bc70 0x7faa3c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.135+0000 7faa5cdb1640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa3c06bc70 0x7faa3c06e130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00a870 tx=0x7faa4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:32.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.139+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa440772f0 con 0x7faa580cec80 2024-02-18T11:47:32.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.310+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faa58074520 con 0x7faa580cec80 2024-02-18T11:47:32.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.499+0000 7faa557fa640 1 -- 172.21.15.116:0/1868560222 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7faa4407a9b0 con 0x7faa580cec80 2024-02-18T11:47:32.491 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa3c06bc70 msgr2=0x7faa3c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa3c06bc70 0x7faa3c06e130 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00a870 tx=0x7faa4c008040 comp rx=0 tx=0).stop 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 msgr2=0x7faa580c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 0x7faa580c9340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa44009890 tx=0x7faa440046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 shutdown_connections 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa3c06bc70 0x7faa3c06e130 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5815dbf0 0x7faa58154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa580cec80 0x7faa580c9340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 --2- 172.21.15.116:0/1868560222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa580b9370 0x7faa5815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 >> 172.21.15.116:0/1868560222 conn(0x7faa580b5480 msgr2=0x7faa580d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:32.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.501+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 shutdown_connections 2024-02-18T11:47:32.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.502+0000 7faa5f015640 1 -- 172.21.15.116:0/1868560222 wait complete. 2024-02-18T11:47:32.508 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-18T11:47:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.623+0000 7fc2b3415640 1 Processor -- start 2024-02-18T11:47:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.623+0000 7fc2b3415640 1 -- start start 2024-02-18T11:47:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.623+0000 7fc2b3415640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2ac0cee50 0x7fc2ac168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.623+0000 7fc2b3415640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ac0d1ee0 con 0x7fc2ac15dbf0 2024-02-18T11:47:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b3415640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ac161dc0 con 0x7fc2ac1617e0 2024-02-18T11:47:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b3415640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ac1620a0 con 0x7fc2ac0cee50 2024-02-18T11:47:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b19b2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2ac0cee50 0x7fc2ac168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.619 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b19b2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2ac0cee50 0x7fc2ac168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51800/0 (socket says 172.21.15.116:51800) 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b11b1640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc2ac15dbf0 0x7fc2ac0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:43038/0 (socket says 172.21.15.116:43038) 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1746455244 learned_addr learned my addr 172.21.15.116:0/1746455244 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1746455244 >> v1:172.21.15.143:6789/0 conn(0x7fc2ac1617e0 legacy=0x7fc2ac0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1746455244 >> v1:172.21.15.116:6789/0 conn(0x7fc2ac15dbf0 legacy=0x7fc2ac0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc29ffff640 1 -- 172.21.15.116:0/1746455244 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720915792 0 0) 0x7fc2ac0d1ee0 con 0x7fc2ac15dbf0 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.624+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1746455244 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac162380 con 0x7fc2ac0cee50 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b19b2640 1 --2- 172.21.15.116:0/1746455244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac0cee50 0x7fc2ac168b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009e50 tx=0x7fc2a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b610d97f40409db server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc29ffff640 1 -- 172.21.15.116:0/1746455244 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a402ca80 con 0x7fc2ac0cee50 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc29ffff640 1 -- 172.21.15.116:0/1746455244 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a402cbe0 con 0x7fc2ac0cee50 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc29ffff640 1 -- 172.21.15.116:0/1746455244 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40343f0 con 0x7fc2ac0cee50 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 -- 172.21.15.116:0/1746455244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac0cee50 msgr2=0x7fc2ac168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1746455244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac0cee50 0x7fc2ac168b10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009e50 tx=0x7fc2a402bee0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 -- 172.21.15.116:0/1746455244 shutdown_connections 2024-02-18T11:47:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1746455244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac0cee50 0x7fc2ac168b10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 -- 172.21.15.116:0/1746455244 >> 172.21.15.116:0/1746455244 conn(0x7fc2ac0b5480 msgr2=0x7fc2ac0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 -- 172.21.15.116:0/1746455244 shutdown_connections 2024-02-18T11:47:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.625+0000 7fc2b3415640 1 -- 172.21.15.116:0/1746455244 wait complete. 2024-02-18T11:47:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 Processor -- start 2024-02-18T11:47:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 -- start start 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc2ac0cee50 0x7fc2ac14df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2ac15dbf0 0x7fc2ac15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 0x7fc2ac15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac150a90 con 0x7fc2ac1617e0 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac159760 con 0x7fc2ac15dbf0 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b3415640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac15f4d0 con 0x7fc2ac0cee50 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b19b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 0x7fc2ac15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b19b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 0x7fc2ac15ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51814/0 (socket says 172.21.15.116:51814) 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.627+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1178158654 learned_addr learned my addr 172.21.15.116:0/1178158654 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b11b1640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc2ac0cee50 0x7fc2ac14df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b09b0640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2ac15dbf0 0x7fc2ac15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc2ac0cee50 msgr2=0x7fc2ac14df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b19b2640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc2ac0cee50 0x7fc2ac14df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2ac15dbf0 msgr2=0x7fc2ac15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b19b2640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2ac15dbf0 0x7fc2ac15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b19b2640 1 -- 172.21.15.116:0/1178158654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4009b30 con 0x7fc2ac1617e0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b19b2640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 0x7fc2ac15ca30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009e50 tx=0x7fc2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4009510 con 0x7fc2ac1617e0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a403e450 con 0x7fc2ac1617e0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac15e150 con 0x7fc2ac1617e0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a403e600 con 0x7fc2ac1617e0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.628+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac1503d0 con 0x7fc2ac1617e0 2024-02-18T11:47:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.629+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac067840 con 0x7fc2ac1617e0 2024-02-18T11:47:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.630+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc2a402cd50 con 0x7fc2ac1617e0 2024-02-18T11:47:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.630+0000 7fc29dffb640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc29006bc70 0x7fc29006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.630+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc2a40acd90 con 0x7fc2ac1617e0 2024-02-18T11:47:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.630+0000 7fc2b11b1640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc29006bc70 0x7fc29006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.630+0000 7fc2b11b1640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc29006bc70 0x7fc29006e130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8000f60 tx=0x7fc2a8007b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.634+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2a4077900 con 0x7fc2ac1617e0 2024-02-18T11:47:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:32.807+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc2ac150230 con 0x7fc2ac1617e0 2024-02-18T11:47:33.493 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.501+0000 7fc29dffb640 1 -- 172.21.15.116:0/1178158654 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc2ac150230 con 0x7fc2ac1617e0 2024-02-18T11:47:33.494 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-02-18T11:47:33.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.506+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc29006bc70 msgr2=0x7fc29006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.506+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc29006bc70 0x7fc29006e130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8000f60 tx=0x7fc2a8007b70 comp rx=0 tx=0).stop 2024-02-18T11:47:33.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.506+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 msgr2=0x7fc2ac15ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.498 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.506+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 0x7fc2ac15ca30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009e50 tx=0x7fc2a4004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.507+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 shutdown_connections 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.507+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc29006bc70 0x7fc29006e130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.507+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2ac1617e0 0x7fc2ac15ca30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.507+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2ac15dbf0 0x7fc2ac15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.507+0000 7fc2b3415640 1 --2- 172.21.15.116:0/1178158654 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc2ac0cee50 0x7fc2ac14df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.507+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 >> 172.21.15.116:0/1178158654 conn(0x7fc2ac0b5480 msgr2=0x7fc2ac0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.508+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 shutdown_connections 2024-02-18T11:47:33.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.508+0000 7fc2b3415640 1 -- 172.21.15.116:0/1178158654 wait complete. 2024-02-18T11:47:33.525 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-18T11:47:33.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.639+0000 7f287513a640 1 Processor -- start 2024-02-18T11:47:33.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.639+0000 7f287513a640 1 -- start start 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f287513a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 0x7f2870161db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f287513a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287006e5c0 con 0x7f287015b840 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f287513a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287006e860 con 0x7f287015f5c0 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f287513a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287006f100 con 0x7f287006e040 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286ed9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f287015b840 0x7f28701616a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:43050/0 (socket says 172.21.15.116:43050) 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286ed9d640 1 -- 172.21.15.116:0/3623497255 learned_addr learned my addr 172.21.15.116:0/3623497255 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286e59c640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 0x7f2870161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792622518 0 0) 0x7f287006e5c0 con 0x7f287015b840 2024-02-18T11:47:33.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2858003520 con 0x7f287015b840 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395119813 0 0) 0x7f2858003520 con 0x7f287015b840 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 msgr2=0x7f2870161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 0x7f2870161db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 >> v1:172.21.15.143:6789/0 conn(0x7f287015f5c0 legacy=0x7f287015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287006eb40 con 0x7f287015b840 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f28600019f0 con 0x7f287015b840 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.640+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 >> v1:172.21.15.116:6789/0 conn(0x7f287015b840 legacy=0x7f28701616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286dd9b640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 0x7f28580039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286dd9b640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2858003f20 0x7f28580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286dd9b640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2858004d00 0x7f2858005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287006f100 con 0x7f2858004d00 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285800ba60 con 0x7f2858003f20 2024-02-18T11:47:33.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285800bc20 con 0x7f287006e040 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286f59e640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2858004d00 0x7f2858005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286ed9d640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 0x7f28580039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286e59c640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2858003f20 0x7f28580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286f59e640 1 -- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 msgr2=0x7f28580039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286f59e640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287006e040 0x7f28580039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286f59e640 1 -- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2858003f20 msgr2=0x7f28580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286f59e640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2858003f20 0x7f28580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.641+0000 7f286f59e640 1 -- 172.21.15.116:0/3623497255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858003520 con 0x7f2858004d00 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f286f59e640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2858004d00 0x7f2858005180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2868001750 tx=0x7f28580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8830f22bea14dbf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680052d0 con 0x7f2858004d00 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286802e050 con 0x7f2858004d00 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f286dd9b640 1 -- 172.21.15.116:0/3623497255 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868003830 con 0x7f2858004d00 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f287513a640 1 -- 172.21.15.116:0/3623497255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2858004d00 msgr2=0x7f2858005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f287513a640 1 --2- 172.21.15.116:0/3623497255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2858004d00 0x7f2858005180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2868001750 tx=0x7f28580057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f286f59e640 1 -- 172.21.15.116:0/3623497255 reap_dead start 2024-02-18T11:47:33.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f287513a640 1 -- 172.21.15.116:0/3623497255 shutdown_connections 2024-02-18T11:47:33.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f287513a640 1 -- 172.21.15.116:0/3623497255 >> 172.21.15.116:0/3623497255 conn(0x7f28700cf9b0 msgr2=0x7f28700cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:33.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f287513a640 1 -- 172.21.15.116:0/3623497255 shutdown_connections 2024-02-18T11:47:33.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.642+0000 7f287513a640 1 -- 172.21.15.116:0/3623497255 wait complete. 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 Processor -- start 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 -- start start 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f287014bae0 0x7f287014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287014d5e0 0x7f28701f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 0x7f28701f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287014e770 con 0x7f287014ec00 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287014db60 con 0x7f287014bae0 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.644+0000 7f287513a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287014de40 con 0x7f287014d5e0 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 0x7f28701f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 0x7f28701f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51824/0 (socket says 172.21.15.116:51824) 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 -- 172.21.15.116:0/698833956 learned_addr learned my addr 172.21.15.116:0/698833956 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286ed9d640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f287014bae0 0x7f287014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286e59c640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287014d5e0 0x7f28701f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 -- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287014d5e0 msgr2=0x7f28701f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287014d5e0 0x7f28701f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 -- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f287014bae0 msgr2=0x7f287014d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f287014bae0 0x7f287014d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 -- 172.21.15.116:0/698833956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287014e120 con 0x7f287014ec00 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286ed9d640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f287014bae0 0x7f287014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f286f59e640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 0x7f28701f3f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2868005040 tx=0x7f2868001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868007a30 con 0x7f287014ec00 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28680067d0 con 0x7f287014ec00 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287014e400 con 0x7f287014ec00 2024-02-18T11:47:33.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868006b60 con 0x7f287014ec00 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.645+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287014bec0 con 0x7f287014ec00 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.646+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870067840 con 0x7f287014ec00 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.647+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f2868006cc0 con 0x7f287014ec00 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.647+0000 7f28577fe640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f285c06bc70 0x7f285c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.647+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f28680abe60 con 0x7f287014ec00 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.647+0000 7f286ed9d640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f285c06bc70 0x7f285c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:33.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.647+0000 7f286ed9d640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f285c06bc70 0x7f285c06e130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2860007260 tx=0x7f2860006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:33.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.652+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28680766a0 con 0x7f287014ec00 2024-02-18T11:47:33.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:33.858+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f287014e590 con 0x7f287014ec00 2024-02-18T11:47:33.977 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:47:33.979 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:47:34.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.517+0000 7f28577fe640 1 -- 172.21.15.116:0/698833956 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2868079d60 con 0x7f287014ec00 2024-02-18T11:47:34.509 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 2 and data pool 3 2024-02-18T11:47:34.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.519+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f285c06bc70 msgr2=0x7f285c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.519+0000 7f287513a640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f285c06bc70 0x7f285c06e130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2860007260 tx=0x7f2860006040 comp rx=0 tx=0).stop 2024-02-18T11:47:34.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.519+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 msgr2=0x7f28701f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.519+0000 7f287513a640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 0x7f28701f3f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2868005040 tx=0x7f2868001120 comp rx=0 tx=0).stop 2024-02-18T11:47:34.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.519+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 shutdown_connections 2024-02-18T11:47:34.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.519+0000 7f287513a640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f285c06bc70 0x7f285c06e130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.520+0000 7f287513a640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f287014ec00 0x7f28701f3f90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.520+0000 7f287513a640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f287014d5e0 0x7f28701f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.520+0000 7f287513a640 1 --2- 172.21.15.116:0/698833956 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f287014bae0 0x7f287014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.520+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 >> 172.21.15.116:0/698833956 conn(0x7f28700cf9b0 msgr2=0x7f287015bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:34.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.520+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 shutdown_connections 2024-02-18T11:47:34.513 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.520+0000 7f287513a640 1 -- 172.21.15.116:0/698833956 wait complete. 2024-02-18T11:47:34.542 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-18T11:47:34.543 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-18T11:47:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.656+0000 7fed96102640 1 Processor -- start 2024-02-18T11:47:34.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.656+0000 7fed96102640 1 -- start start 2024-02-18T11:47:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed96102640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 0x7fed900d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed96102640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed900b58a0 con 0x7fed90070c00 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed96102640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9015c500 con 0x7fed901452e0 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed96102640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed901456c0 con 0x7fed900b6440 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8effd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fed90070c00 0x7fed900b5d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:43058/0 (socket says 172.21.15.116:43058) 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8effd640 1 -- 172.21.15.116:0/277830386 learned_addr learned my addr 172.21.15.116:0/277830386 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907183813 0 0) 0x7fed900b58a0 con 0x7fed90070c00 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed78003520 con 0x7fed90070c00 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8ffff640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 0x7fed900d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156542443 0 0) 0x7fed78003520 con 0x7fed90070c00 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 msgr2=0x7fed900d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 0x7fed900d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 >> v1:172.21.15.143:6789/0 conn(0x7fed901452e0 legacy=0x7fed9015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed900b4c90 con 0x7fed90070c00 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fed84001670 con 0x7fed90070c00 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 >> v1:172.21.15.116:6789/0 conn(0x7fed90070c00 legacy=0x7fed900b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed780047c0 0x7fed78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 0x7fed78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed901456c0 con 0x7fed780039a0 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7800ba60 con 0x7fed780047c0 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7800bc20 con 0x7fed900b6440 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8effd640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed780047c0 0x7fed78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.657+0000 7fed8ffff640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 0x7fed78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8f7fe640 1 -- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 msgr2=0x7fed78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900b6440 0x7fed78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8f7fe640 1 -- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed780047c0 msgr2=0x7fed78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed780047c0 0x7fed78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8f7fe640 1 -- 172.21.15.116:0/277830386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed78003520 con 0x7fed780039a0 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8effd640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed780047c0 0x7fed78004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed78003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fed80001720 tx=0x7fed780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351cf5fe2c4110d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800052d0 con 0x7fed780039a0 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8002e050 con 0x7fed780039a0 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.658+0000 7fed8e7fc640 1 -- 172.21.15.116:0/277830386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800035f0 con 0x7fed780039a0 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed96102640 1 -- 172.21.15.116:0/277830386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 msgr2=0x7fed78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed96102640 1 --2- 172.21.15.116:0/277830386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed78003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fed80001720 tx=0x7fed780057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed8ffff640 1 -- 172.21.15.116:0/277830386 reap_dead start 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed96102640 1 -- 172.21.15.116:0/277830386 shutdown_connections 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed96102640 1 -- 172.21.15.116:0/277830386 >> 172.21.15.116:0/277830386 conn(0x7fed9006f460 msgr2=0x7fed9006f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed96102640 1 -- 172.21.15.116:0/277830386 shutdown_connections 2024-02-18T11:47:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.659+0000 7fed96102640 1 -- 172.21.15.116:0/277830386 wait complete. 2024-02-18T11:47:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.660+0000 7fed96102640 1 Processor -- start 2024-02-18T11:47:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.660+0000 7fed96102640 1 -- start start 2024-02-18T11:47:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed96102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed900afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed96102640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900ab7e0 0x7fed900abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed96102640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed9015d410 0x7fed9015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed96102640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9014b600 con 0x7fed780039a0 2024-02-18T11:47:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed96102640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed900ac1a0 con 0x7fed9015d410 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed96102640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9015e460 con 0x7fed900ab7e0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed900afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed900afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51840/0 (socket says 172.21.15.116:51840) 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 -- 172.21.15.116:0/1660106288 learned_addr learned my addr 172.21.15.116:0/1660106288 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8ffff640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed9015d410 0x7fed9015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8effd640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900ab7e0 0x7fed900abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 -- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900ab7e0 msgr2=0x7fed900abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900ab7e0 0x7fed900abc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 -- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed9015d410 msgr2=0x7fed9015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed9015d410 0x7fed9015d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.661+0000 7fed8f7fe640 1 -- 172.21.15.116:0/1660106288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9015e180 con 0x7fed780039a0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.662+0000 7fed8f7fe640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed900afdc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed80008fd0 tx=0x7fed80001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.662+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed800076c0 con 0x7fed780039a0 2024-02-18T11:47:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.662+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80007a30 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.662+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80006c00 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.662+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9014af40 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.662+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed900b0300 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.663+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed90074520 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.663+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fed800035f0 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.664+0000 7fed77fff640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fed7c06bc70 0x7fed7c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.664+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed800ac4a0 con 0x7fed780039a0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.664+0000 7fed8effd640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fed7c06bc70 0x7fed7c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.664+0000 7fed8effd640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fed7c06bc70 0x7fed7c06e130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed900a8d80 tx=0x7fed84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.668+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed80076d10 con 0x7fed780039a0 2024-02-18T11:47:34.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:34.846+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fed900d1070 con 0x7fed780039a0 2024-02-18T11:47:35.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.530+0000 7fed77fff640 1 -- 172.21.15.116:0/1660106288 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fed8007a3d0 con 0x7fed780039a0 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.533+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fed7c06bc70 msgr2=0x7fed7c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.533+0000 7fed96102640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fed7c06bc70 0x7fed7c06e130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fed900a8d80 tx=0x7fed84009040 comp rx=0 tx=0).stop 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 msgr2=0x7fed900afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed900afdc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed80008fd0 tx=0x7fed80001120 comp rx=0 tx=0).stop 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 shutdown_connections 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fed7c06bc70 0x7fed7c06e130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed9015d410 0x7fed9015d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed900ab7e0 0x7fed900abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 --2- 172.21.15.116:0/1660106288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fed780039a0 0x7fed900afdc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 >> 172.21.15.116:0/1660106288 conn(0x7fed9006f460 msgr2=0x7fed900cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:35.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 shutdown_connections 2024-02-18T11:47:35.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.534+0000 7fed96102640 1 -- 172.21.15.116:0/1660106288 wait complete. 2024-02-18T11:47:35.538 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-18T11:47:35.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.655+0000 7f3ccf6df640 1 Processor -- start 2024-02-18T11:47:35.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.655+0000 7f3ccf6df640 1 -- start start 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccf6df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3cc80af8a0 0x7f3cc80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccf6df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc80afc80 0x7f3cc80cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccf6df640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3cc806b700 0x7f3cc806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccf6df640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80cce80 con 0x7f3cc806b700 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccf6df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80c6cc0 con 0x7f3cc80af8a0 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccf6df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80c6fa0 con 0x7f3cc80afc80 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3cc806b700 0x7f3cc806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3cc806b700 0x7f3cc806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51852/0 (socket says 172.21.15.116:51852) 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/3444470393 learned_addr learned my addr 172.21.15.116:0/3444470393 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3cccc7a640 1 --2- 172.21.15.116:0/3444470393 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc80afc80 0x7f3cc80cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:35.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccd47b640 1 --2- 172.21.15.116:0/3444470393 >> v2:172.21.15.143:3300/0 conn(0x7f3cc80af8a0 0x7f3cc80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:35.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/3444470393 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc80afc80 msgr2=0x7f3cc80cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:35.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 --2- 172.21.15.116:0/3444470393 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc80afc80 0x7f3cc80cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/3444470393 >> v2:172.21.15.143:3300/0 conn(0x7f3cc80af8a0 msgr2=0x7f3cc80b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:35.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 --2- 172.21.15.116:0/3444470393 >> v2:172.21.15.143:3300/0 conn(0x7f3cc80af8a0 0x7f3cc80b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.656+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/3444470393 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc80c7280 con 0x7f3cc806b700 2024-02-18T11:47:35.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.657+0000 7f3ccdc7c640 1 --2- 172.21.15.116:0/3444470393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc806b700 0x7f3cc806ab60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8070aa0 tx=0x7f3cc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aae7e8601318ef1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:35.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.657+0000 7f3cbffff640 1 -- 172.21.15.116:0/3444470393 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc002ca70 con 0x7f3cc806b700 2024-02-18T11:47:35.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.657+0000 7f3cbffff640 1 -- 172.21.15.116:0/3444470393 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc002cbd0 con 0x7f3cc806b700 2024-02-18T11:47:35.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.657+0000 7f3cbffff640 1 -- 172.21.15.116:0/3444470393 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00343f0 con 0x7f3cc806b700 2024-02-18T11:47:35.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 -- 172.21.15.116:0/3444470393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc806b700 msgr2=0x7f3cc806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/3444470393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc806b700 0x7f3cc806ab60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8070aa0 tx=0x7f3cc002bed0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 -- 172.21.15.116:0/3444470393 shutdown_connections 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/3444470393 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc80afc80 0x7f3cc80cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/3444470393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc806b700 0x7f3cc806ab60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/3444470393 >> v2:172.21.15.143:3300/0 conn(0x7f3cc80af8a0 0x7f3cc80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 -- 172.21.15.116:0/3444470393 >> 172.21.15.116:0/3444470393 conn(0x7f3cc80b2230 msgr2=0x7f3cc80b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 -- 172.21.15.116:0/3444470393 shutdown_connections 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.658+0000 7f3ccf6df640 1 -- 172.21.15.116:0/3444470393 wait complete. 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.659+0000 7f3ccf6df640 1 Processor -- start 2024-02-18T11:47:35.652 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.659+0000 7f3ccf6df640 1 -- start start 2024-02-18T11:47:35.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc806b700 0x7f3cc809a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc80af8a0 0x7f3cc8161e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 0x7f3cc80adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b5840 con 0x7f3cc80afc80 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b5b20 con 0x7f3cc80af8a0 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b5e00 con 0x7f3cc806b700 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 0x7f3cc80adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 0x7f3cc80adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51864/0 (socket says 172.21.15.116:51864) 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/2204937131 learned_addr learned my addr 172.21.15.116:0/2204937131 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccd47b640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc806b700 0x7f3cc809a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc806b700 msgr2=0x7f3cc809a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc806b700 0x7f3cc809a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc80af8a0 msgr2=0x7f3cc8161e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc80af8a0 0x7f3cc8161e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 -- 172.21.15.116:0/2204937131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0009b30 con 0x7f3cc80afc80 2024-02-18T11:47:35.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccdc7c640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 0x7f3cc80adaf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0036040 tx=0x7f3cc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0009510 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc003e450 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc80b60e0 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.660+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc003e800 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.661+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc80b54d0 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.661+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc81f66c0 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.662+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3cc003e960 con 0x7f3cc80afc80 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.662+0000 7f3cbdffb640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3cac06bc70 0x7f3cac06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.662+0000 7f3ccd47b640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3cac06bc70 0x7f3cac06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:35.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.662+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f3cc00ad650 con 0x7f3cc80afc80 2024-02-18T11:47:35.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.663+0000 7f3ccd47b640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3cac06bc70 0x7f3cac06e130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:35.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.667+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cc81f66c0 con 0x7f3cc80afc80 2024-02-18T11:47:35.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:35.839+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3cc81f66c0 con 0x7f3cc80afc80 2024-02-18T11:47:37.539 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.546+0000 7f3cbdffb640 1 -- 172.21.15.116:0/2204937131 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3cc81f66c0 con 0x7f3cc80afc80 2024-02-18T11:47:37.539 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data_ec' created 2024-02-18T11:47:37.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.551+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3cac06bc70 msgr2=0x7f3cac06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.551+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3cac06bc70 0x7f3cac06e130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4006fd0 tx=0x7f3cc4008040 comp rx=0 tx=0).stop 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.551+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 msgr2=0x7f3cc80adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.551+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 0x7f3cc80adaf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0036040 tx=0x7f3cc0004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.552+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 shutdown_connections 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.552+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3cac06bc70 0x7f3cac06e130 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.552+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3cc80afc80 0x7f3cc80adaf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.552+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3cc80af8a0 0x7f3cc8161e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.552+0000 7f3ccf6df640 1 --2- 172.21.15.116:0/2204937131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3cc806b700 0x7f3cc809a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.553+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 >> 172.21.15.116:0/2204937131 conn(0x7f3cc80b2230 msgr2=0x7f3cc80b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:37.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.553+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 shutdown_connections 2024-02-18T11:47:37.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.555+0000 7f3ccf6df640 1 -- 172.21.15.116:0/2204937131 wait complete. 2024-02-18T11:47:37.563 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-18T11:47:37.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.678+0000 7f776fd8c640 1 Processor -- start 2024-02-18T11:47:37.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.678+0000 7f776fd8c640 1 -- start start 2024-02-18T11:47:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776fd8c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7768161fd0 0x7f776815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776fd8c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7768099a50 con 0x7f77680cd610 2024-02-18T11:47:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776fd8c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7768099ca0 con 0x7f77680d0a80 2024-02-18T11:47:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776fd8c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7768099f80 con 0x7f7768161fd0 2024-02-18T11:47:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776db28640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f77680cd610 0x7f77680d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50056/0 (socket says 172.21.15.116:50056) 2024-02-18T11:47:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776db28640 1 -- 172.21.15.116:0/3264079384 learned_addr learned my addr 172.21.15.116:0/3264079384 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.679+0000 7f776e329640 1 --2- 172.21.15.116:0/3264079384 >> v2:172.21.15.143:3300/0 conn(0x7f7768161fd0 0x7f776815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888153979 0 0) 0x7f7768099a50 con 0x7f77680cd610 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7754003520 con 0x7f77680cd610 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586840120 0 0) 0x7f7754003520 con 0x7f77680cd610 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 >> v2:172.21.15.143:3300/0 conn(0x7f7768161fd0 msgr2=0x7f776815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 --2- 172.21.15.116:0/3264079384 >> v2:172.21.15.143:3300/0 conn(0x7f7768161fd0 0x7f776815d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 >> v1:172.21.15.143:6789/0 conn(0x7f77680d0a80 legacy=0x7f77681618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776809a260 con 0x7f77680cd610 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f77640019f0 con 0x7f77680cd610 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776e329640 1 --2- 172.21.15.116:0/3264079384 >> v2:172.21.15.143:3300/0 conn(0x7f7768161fd0 0x7f776815d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 >> v1:172.21.15.116:6789/0 conn(0x7f77680cd610 legacy=0x7f77680d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7754003970 0x7f7754003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7754004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7754005180 0x7f775400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7768099f80 con 0x7f77540047c0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775400c030 con 0x7f7754003970 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775400c1f0 con 0x7f7754005180 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776d327640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7754004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776db28640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7754003970 0x7f7754003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776e329640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7754005180 0x7f775400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776d327640 1 -- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7754005180 msgr2=0x7f775400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776d327640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7754005180 0x7f775400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776d327640 1 -- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7754003970 msgr2=0x7f7754003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776d327640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7754003970 0x7f7754003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776d327640 1 -- 172.21.15.116:0/3264079384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7754003520 con 0x7f77540047c0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.680+0000 7f776e329640 1 -- 172.21.15.116:0/3264079384 reap_dead start 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.681+0000 7f776d327640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7754004c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7758001750 tx=0x7f77540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18182e93d6d38d26 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.681+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77580052d0 con 0x7f77540047c0 2024-02-18T11:47:37.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.681+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775802e050 con 0x7f77540047c0 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.681+0000 7f776cb26640 1 -- 172.21.15.116:0/3264079384 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758003830 con 0x7f77540047c0 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 -- 172.21.15.116:0/3264079384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 msgr2=0x7f7754004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7754004c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7758001750 tx=0x7f77540057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 -- 172.21.15.116:0/3264079384 shutdown_connections 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 --2- 172.21.15.116:0/3264079384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7754004c40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 -- 172.21.15.116:0/3264079384 >> 172.21.15.116:0/3264079384 conn(0x7f77680b31f0 msgr2=0x7f77680b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 -- 172.21.15.116:0/3264079384 shutdown_connections 2024-02-18T11:47:37.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.682+0000 7f776fd8c640 1 -- 172.21.15.116:0/3264079384 wait complete. 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.683+0000 7f776fd8c640 1 Processor -- start 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.683+0000 7f776fd8c640 1 -- start start 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7768154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776809b700 0x7f776814e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7768150ac0 0x7f7768150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680b6dc0 con 0x7f77540047c0 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77681514b0 con 0x7f776809b700 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776814e7c0 con 0x7f7768150ac0 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7768154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7768154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34392/0 (socket says 172.21.15.116:34392) 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 -- 172.21.15.116:0/2678460694 learned_addr learned my addr 172.21.15.116:0/2678460694 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776d327640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776809b700 0x7f776814e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776e329640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7768150ac0 0x7f7768150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 -- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7768150ac0 msgr2=0x7f7768150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7768150ac0 0x7f7768150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 -- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776809b700 msgr2=0x7f776814e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776809b700 0x7f776814e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 -- 172.21.15.116:0/2678460694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680b6140 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776e329640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7768150ac0 0x7f7768150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776db28640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7768154b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77640079d0 tx=0x7f776400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764007ac0 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77640133f0 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77680b6420 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764013780 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.684+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77680b69e0 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.685+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7768067840 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.686+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f77640138e0 con 0x7f77540047c0 2024-02-18T11:47:37.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.686+0000 7f775e7fc640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f774c06bca0 0x7f774c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:37.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.686+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f776408b320 con 0x7f77540047c0 2024-02-18T11:47:37.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.686+0000 7f776d327640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f774c06bca0 0x7f774c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:37.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.687+0000 7f776d327640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f774c06bca0 0x7f774c06e160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7758033000 tx=0x7f7758009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:37.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.691+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7764055870 con 0x7f77540047c0 2024-02-18T11:47:37.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:37.860+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f776814ea70 con 0x7f77540047c0 2024-02-18T11:47:38.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.551+0000 7f775e7fc640 1 -- 172.21.15.116:0/2678460694 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f776814ea70 con 0x7f77540047c0 2024-02-18T11:47:38.543 INFO:teuthology.orchestra.run.smithi116.stderr:set pool 4 allow_ec_overwrites to true 2024-02-18T11:47:38.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.555+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f774c06bca0 msgr2=0x7f774c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:38.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.555+0000 7f776fd8c640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f774c06bca0 0x7f774c06e160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7758033000 tx=0x7f7758009030 comp rx=0 tx=0).stop 2024-02-18T11:47:38.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.555+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 msgr2=0x7f7768154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:38.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.555+0000 7f776fd8c640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7768154b10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77640079d0 tx=0x7f776400a6e0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.558+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 shutdown_connections 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.558+0000 7f776fd8c640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f774c06bca0 0x7f774c06e160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.558+0000 7f776fd8c640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7768150ac0 0x7f7768150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.558+0000 7f776fd8c640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776809b700 0x7f776814e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.558+0000 7f776fd8c640 1 --2- 172.21.15.116:0/2678460694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77540047c0 0x7f7768154b10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.559+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 >> 172.21.15.116:0/2678460694 conn(0x7f77680b31f0 msgr2=0x7f7768070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.565+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 shutdown_connections 2024-02-18T11:47:38.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.566+0000 7f776fd8c640 1 -- 172.21.15.116:0/2678460694 wait complete. 2024-02-18T11:47:38.576 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-18T11:47:38.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.686+0000 7f004e2f4640 1 Processor -- start 2024-02-18T11:47:38.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.686+0000 7f004e2f4640 1 -- start start 2024-02-18T11:47:38.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004e2f4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 0x7f00481618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004e2f4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 0x7f0048161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004e2f4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0048099a50 con 0x7f00480cd610 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004e2f4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0048099ca0 con 0x7f00480d0a80 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004e2f4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0048099f80 con 0x7f004809ad40 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f00477fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 0x7f00481618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 0x7f0048161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 0x7f0048161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:40616/0 (socket says 172.21.15.116:40616) 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f00477fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 0x7f00481618c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:58142/0 (socket says 172.21.15.116:58142) 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 -- 172.21.15.116:0/2577600147 learned_addr learned my addr 172.21.15.116:0/2577600147 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f0047fff640 1 --1- 172.21.15.116:0/2577600147 >> v1:172.21.15.143:6789/0 conn(0x7f00480cd610 0x7f00480d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:37592/0 (socket says 172.21.15.116:37592) 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 -- 172.21.15.116:0/2577600147 >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 msgr2=0x7f00481618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 --2- 172.21.15.116:0/2577600147 >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 0x7f00481618c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 -- 172.21.15.116:0/2577600147 >> v1:172.21.15.143:6789/0 conn(0x7f00480cd610 legacy=0x7f00480d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f004c891640 1 -- 172.21.15.116:0/2577600147 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004809a260 con 0x7f004809ad40 2024-02-18T11:47:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.687+0000 7f00477fe640 1 --2- 172.21.15.116:0/2577600147 >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 0x7f00481618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.688+0000 7f004c891640 1 --2- 172.21.15.116:0/2577600147 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 0x7f0048161e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0040009e50 tx=0x7f004002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=309359247136272d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.688+0000 7f0046ffd640 1 -- 172.21.15.116:0/2577600147 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004002cb90 con 0x7f004809ad40 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.688+0000 7f0046ffd640 1 -- 172.21.15.116:0/2577600147 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004002ccf0 con 0x7f004809ad40 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.688+0000 7f0046ffd640 1 -- 172.21.15.116:0/2577600147 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040034610 con 0x7f004809ad40 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 -- 172.21.15.116:0/2577600147 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 msgr2=0x7f0048161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2577600147 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 0x7f0048161e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0040009e50 tx=0x7f004002bff0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 -- 172.21.15.116:0/2577600147 shutdown_connections 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2577600147 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f004809ad40 0x7f0048161e00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2577600147 >> v2:172.21.15.143:3300/0 conn(0x7f00480d0a80 0x7f00481618c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 -- 172.21.15.116:0/2577600147 >> 172.21.15.116:0/2577600147 conn(0x7f00480b31f0 msgr2=0x7f00480b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:38.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 -- 172.21.15.116:0/2577600147 shutdown_connections 2024-02-18T11:47:38.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.689+0000 7f004e2f4640 1 -- 172.21.15.116:0/2577600147 wait complete. 2024-02-18T11:47:38.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.690+0000 7f004e2f4640 1 Processor -- start 2024-02-18T11:47:38.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.690+0000 7f004e2f4640 1 -- start start 2024-02-18T11:47:38.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004e2f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f004809ad40 0x7f00481f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:38.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004e2f4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00480cd610 0x7f004814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004e2f4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 0x7f004806ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004e2f4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00481f3cb0 con 0x7f00480d0a80 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004e2f4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0048150130 con 0x7f004809ad40 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004e2f4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00480c6390 con 0x7f00480cd610 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 0x7f004806ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 0x7f004806ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34400/0 (socket says 172.21.15.116:34400) 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 -- 172.21.15.116:0/2737317422 learned_addr learned my addr 172.21.15.116:0/2737317422 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f00477fe640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00480cd610 0x7f004814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f0047fff640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f004809ad40 0x7f00481f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 -- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00480cd610 msgr2=0x7f004814fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00480cd610 0x7f004814fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 -- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f004809ad40 msgr2=0x7f00481f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f004809ad40 0x7f00481f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 -- 172.21.15.116:0/2737317422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009400 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.691+0000 7f004c891640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 0x7f004806ff10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0040036040 tx=0x7f0040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.692+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040009510 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.692+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004003e450 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.692+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0040009b30 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.692+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004003e820 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.692+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00480c6d90 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.693+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0048067840 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.693+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f004003e980 con 0x7f00480d0a80 2024-02-18T11:47:38.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.693+0000 7f0044ff9640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f003406bc70 0x7f003406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:38.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.693+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f00400ad3b0 con 0x7f00480d0a80 2024-02-18T11:47:38.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.694+0000 7f0047fff640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f003406bc70 0x7f003406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:38.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.694+0000 7f0047fff640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f003406bc70 0x7f003406e130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0038007fd0 tx=0x7f0038007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:38.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.698+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00400779b0 con 0x7f00480d0a80 2024-02-18T11:47:38.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:38.906+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f0048074520 con 0x7f00480d0a80 2024-02-18T11:47:38.931 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T11:47:38.939+0000 7fb1126b5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-18T11:47:39.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.559+0000 7f0044ff9640 1 -- 172.21.15.116:0/2737317422 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f004002ce60 con 0x7f00480d0a80 2024-02-18T11:47:39.551 INFO:teuthology.orchestra.run.smithi116.stderr:added data pool 4 to fsmap 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f003406bc70 msgr2=0x7f003406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f003406bc70 0x7f003406e130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0038007fd0 tx=0x7f0038007e80 comp rx=0 tx=0).stop 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 msgr2=0x7f004806ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 0x7f004806ff10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0040036040 tx=0x7f0040004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 shutdown_connections 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f003406bc70 0x7f003406e130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00480d0a80 0x7f004806ff10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00480cd610 0x7f004814fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 --2- 172.21.15.116:0/2737317422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f004809ad40 0x7f00481f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 >> 172.21.15.116:0/2737317422 conn(0x7f00480b31f0 msgr2=0x7f00480b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 shutdown_connections 2024-02-18T11:47:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.561+0000 7f004e2f4640 1 -- 172.21.15.116:0/2737317422 wait complete. 2024-02-18T11:47:39.566 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:47:39.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.680+0000 7f7f1b13a640 1 Processor -- start 2024-02-18T11:47:39.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.680+0000 7f7f1b13a640 1 -- start start 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.681+0000 7f7f1b13a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 0x7f7f1415df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.681+0000 7f7f1b13a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f140b3940 0x7f7f1406ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.681+0000 7f7f1b13a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1406da20 con 0x7f7f1415b5e0 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.681+0000 7f7f1b13a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1406f860 con 0x7f7f140b3940 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.681+0000 7f7f1b13a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406dc50 con 0x7f7f140b2db0 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.681+0000 7f7f13fff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 0x7f7f1415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f196d7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f140b3940 0x7f7f1406ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:39.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f13fff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 0x7f7f1415df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:40636/0 (socket says 172.21.15.116:40636) 2024-02-18T11:47:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f13fff640 1 -- 172.21.15.116:0/495091270 learned_addr learned my addr 172.21.15.116:0/495091270 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f13fff640 1 -- 172.21.15.116:0/495091270 >> v2:172.21.15.143:3300/0 conn(0x7f7f140b3940 msgr2=0x7f7f1406ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f13fff640 1 --2- 172.21.15.116:0/495091270 >> v2:172.21.15.143:3300/0 conn(0x7f7f140b3940 0x7f7f1406ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f13fff640 1 -- 172.21.15.116:0/495091270 >> v1:172.21.15.143:6789/0 conn(0x7f7f1415b5e0 legacy=0x7f7f140b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f13fff640 1 -- 172.21.15.116:0/495091270 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1406df30 con 0x7f7f140b2db0 2024-02-18T11:47:39.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.682+0000 7f7f196d7640 1 --2- 172.21.15.116:0/495091270 >> v2:172.21.15.143:3300/0 conn(0x7f7f140b3940 0x7f7f1406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:39.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f13fff640 1 --2- 172.21.15.116:0/495091270 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 0x7f7f1415df20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009870 tx=0x7f7f0802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6e7f91af3b0028 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f137fe640 1 -- 172.21.15.116:0/495091270 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08030840 con 0x7f7f140b2db0 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f137fe640 1 -- 172.21.15.116:0/495091270 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f080309a0 con 0x7f7f140b2db0 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f137fe640 1 -- 172.21.15.116:0/495091270 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08030d30 con 0x7f7f140b2db0 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 -- 172.21.15.116:0/495091270 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 msgr2=0x7f7f1415df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/495091270 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 0x7f7f1415df20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009870 tx=0x7f7f0802b980 comp rx=0 tx=0).stop 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 -- 172.21.15.116:0/495091270 shutdown_connections 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/495091270 >> v2:172.21.15.143:3300/0 conn(0x7f7f140b3940 0x7f7f1406ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/495091270 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f140b2db0 0x7f7f1415df20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 -- 172.21.15.116:0/495091270 >> 172.21.15.116:0/495091270 conn(0x7f7f140b23d0 msgr2=0x7f7f140cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 -- 172.21.15.116:0/495091270 shutdown_connections 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.683+0000 7f7f1b13a640 1 -- 172.21.15.116:0/495091270 wait complete. 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 Processor -- start 2024-02-18T11:47:39.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 -- start start 2024-02-18T11:47:39.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 0x7f7f140ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f140b3940 0x7f7f14149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f1415b5e0 0x7f7f140ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14157f00 con 0x7f7f140b2db0 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14157c20 con 0x7f7f140b3940 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.685+0000 7f7f1b13a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140ad940 con 0x7f7f1415b5e0 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 0x7f7f140ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 0x7f7f140ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34406/0 (socket says 172.21.15.116:34406) 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 -- 172.21.15.116:0/3782639984 learned_addr learned my addr 172.21.15.116:0/3782639984 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f196d7640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f1415b5e0 0x7f7f140ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f13fff640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f140b3940 0x7f7f14149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 -- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f1415b5e0 msgr2=0x7f7f140ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f1415b5e0 0x7f7f140ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 -- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f140b3940 msgr2=0x7f7f14149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f140b3940 0x7f7f14149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 -- 172.21.15.116:0/3782639984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009a70 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f13fff640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f140b3940 0x7f7f14149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f18ed6640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 0x7f7f140ae420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c3c0 tx=0x7f7f0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040052d0 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04005450 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f080094d0 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04007260 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.686+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f140adf00 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.687+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f140ad7a0 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.688+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7f040073c0 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.688+0000 7f7f117fa640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f0006bc70 0x7f7f0006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.688+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7f04089e80 con 0x7f7f140b2db0 2024-02-18T11:47:39.683 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.688+0000 7f7f13fff640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f0006bc70 0x7f7f0006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:39.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.688+0000 7f7f13fff640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f0006bc70 0x7f7f0006e130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7f08036000 tx=0x7f7f0803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:39.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.692+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f140ad7a0 con 0x7f7f140b2db0 2024-02-18T11:47:39.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.902+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f140ad7a0 con 0x7f7f140b2db0 2024-02-18T11:47:39.894 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.902+0000 7f7f117fa640 1 -- 172.21.15.116:0/3782639984 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f7f140ad7a0 con 0x7f7f140b2db0 2024-02-18T11:47:39.895 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:39.895 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:38.907664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-18T11:47:39.895 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f0006bc70 msgr2=0x7f7f0006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f0006bc70 0x7f7f0006e130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7f08036000 tx=0x7f7f0803b040 comp rx=0 tx=0).stop 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 msgr2=0x7f7f140ae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 0x7f7f140ae420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c3c0 tx=0x7f7f0400c880 comp rx=0 tx=0).stop 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 shutdown_connections 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f0006bc70 0x7f7f0006e130 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f1415b5e0 0x7f7f140ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f140b3940 0x7f7f14149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.897 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 --2- 172.21.15.116:0/3782639984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f140b2db0 0x7f7f140ae420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:39.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 >> 172.21.15.116:0/3782639984 conn(0x7f7f140b23d0 msgr2=0x7f7f14160470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:39.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 shutdown_connections 2024-02-18T11:47:39.898 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:39.904+0000 7f7f1b13a640 1 -- 172.21.15.116:0/3782639984 wait complete. 2024-02-18T11:47:39.907 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:40.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.015+0000 7f4169c39640 1 Processor -- start 2024-02-18T11:47:40.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.015+0000 7f4169c39640 1 -- start start 2024-02-18T11:47:40.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f4169c39640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f41640cc2a0 0x7f416409a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f4169c39640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41640cc720 con 0x7f416415cde0 2024-02-18T11:47:40.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f4169c39640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416415d1c0 con 0x7f41640cf360 2024-02-18T11:47:40.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f4169c39640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41640cd460 con 0x7f41640cc2a0 2024-02-18T11:47:40.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41637fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f416415cde0 0x7f416415d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50062/0 (socket says 172.21.15.116:50062) 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41637fe640 1 -- 172.21.15.116:0/612530170 learned_addr learned my addr 172.21.15.116:0/612530170 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373401581 0 0) 0x7f41640cc720 con 0x7f416415cde0 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f414c003520 con 0x7f416415cde0 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f4162ffd640 1 --2- 172.21.15.116:0/612530170 >> v2:172.21.15.143:3300/0 conn(0x7f41640cc2a0 0x7f416409a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2916812171 0 0) 0x7f414c003520 con 0x7f416415cde0 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 >> v2:172.21.15.143:3300/0 conn(0x7f41640cc2a0 msgr2=0x7f416409a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 --2- 172.21.15.116:0/612530170 >> v2:172.21.15.143:3300/0 conn(0x7f41640cc2a0 0x7f416409a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 >> v1:172.21.15.143:6789/0 conn(0x7f41640cf360 legacy=0x7f41640cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164099bf0 con 0x7f416415cde0 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f4154001670 con 0x7f416415cde0 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 >> v1:172.21.15.116:6789/0 conn(0x7f416415cde0 legacy=0x7f416415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c003970 0x7f414c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f414c0047c0 0x7f414c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f414c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640cd460 con 0x7f414c005180 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414c00c0b0 con 0x7f414c003970 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.016+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414c00c270 con 0x7f414c0047c0 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f414c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f41637fe640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c003970 0x7f414c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4162ffd640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f414c0047c0 0x7f414c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 -- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f414c0047c0 msgr2=0x7f414c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f414c0047c0 0x7f414c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 -- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c003970 msgr2=0x7f414c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f414c003970 0x7f414c003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 -- 172.21.15.116:0/612530170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c003520 con 0x7f414c005180 2024-02-18T11:47:40.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4162ffd640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f414c0047c0 0x7f414c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.017+0000 7f4163fff640 1 -- 172.21.15.116:0/612530170 reap_dead start 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4163fff640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f414c00b980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41640cf7c0 tx=0x7f414c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d23cc18bdcb305d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c006d50 con 0x7f414c005180 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c034ad0 con 0x7f414c005180 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f41627fc640 1 -- 172.21.15.116:0/612530170 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c034e80 con 0x7f414c005180 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 -- 172.21.15.116:0/612530170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 msgr2=0x7f414c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f414c00b980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f41640cf7c0 tx=0x7f414c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 -- 172.21.15.116:0/612530170 shutdown_connections 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 --2- 172.21.15.116:0/612530170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f414c00b980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 -- 172.21.15.116:0/612530170 >> 172.21.15.116:0/612530170 conn(0x7f41640d1790 msgr2=0x7f41640d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 -- 172.21.15.116:0/612530170 shutdown_connections 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.018+0000 7f4169c39640 1 -- 172.21.15.116:0/612530170 wait complete. 2024-02-18T11:47:40.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.019+0000 7f4169c39640 1 Processor -- start 2024-02-18T11:47:40.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 -- start start 2024-02-18T11:47:40.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f4164156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f41641ef630 0x7f41640c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41640c9ae0 0x7f416406ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41641f2a00 con 0x7f414c005180 2024-02-18T11:47:40.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41640ca020 con 0x7f41640c9ae0 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f4169c39640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41640b5da0 con 0x7f41641ef630 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f41637fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f4164156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f41637fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f4164156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34426/0 (socket says 172.21.15.116:34426) 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.020+0000 7f41637fe640 1 -- 172.21.15.116:0/2031128329 learned_addr learned my addr 172.21.15.116:0/2031128329 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f4163fff640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41640c9ae0 0x7f416406ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f4162ffd640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f41641ef630 0x7f41640c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f41637fe640 1 -- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f41641ef630 msgr2=0x7f41640c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f41637fe640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f41641ef630 0x7f41640c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f41637fe640 1 -- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41640c9ae0 msgr2=0x7f416406ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f41637fe640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41640c9ae0 0x7f416406ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f41637fe640 1 -- 172.21.15.116:0/2031128329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640b6080 con 0x7f414c005180 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f4162ffd640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f41641ef630 0x7f41640c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f41637fe640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f4164156830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41540072b0 tx=0x7f4154007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400e8f0 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415400ea50 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415400ee00 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41640b6360 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.021+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41640ca300 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.022+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4164067840 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.022+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4154016070 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.023+0000 7f414bfff640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f415006bc70 0x7f415006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.023+0000 7f4162ffd640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f415006bc70 0x7f415006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.023+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f415408a730 con 0x7f414c005180 2024-02-18T11:47:40.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.023+0000 7f4162ffd640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f415006bc70 0x7f415006e130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41640c8910 tx=0x7f4158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.027+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4154054c60 con 0x7f414c005180 2024-02-18T11:47:40.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.199+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41640cbbd0 con 0x7f414c005180 2024-02-18T11:47:40.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.200+0000 7f414bfff640 1 -- 172.21.15.116:0/2031128329 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f415401e090 con 0x7f414c005180 2024-02-18T11:47:40.193 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:40.193 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:40.194 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f415006bc70 msgr2=0x7f415006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f415006bc70 0x7f415006e130 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41640c8910 tx=0x7f4158008040 comp rx=0 tx=0).stop 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 msgr2=0x7f4164156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f4164156830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41540072b0 tx=0x7f4154007770 comp rx=0 tx=0).stop 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 shutdown_connections 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f415006bc70 0x7f415006e130 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41640c9ae0 0x7f416406ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f41641ef630 0x7f41640c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 --2- 172.21.15.116:0/2031128329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f414c005180 0x7f4164156830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 >> 172.21.15.116:0/2031128329 conn(0x7f41640d1790 msgr2=0x7f416415bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 shutdown_connections 2024-02-18T11:47:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.202+0000 7f4169c39640 1 -- 172.21.15.116:0/2031128329 wait complete. 2024-02-18T11:47:40.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:40.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.314+0000 7f683153e640 1 Processor -- start 2024-02-18T11:47:40.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.314+0000 7f683153e640 1 -- start start 2024-02-18T11:47:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f683153e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f682c15dc60 0x7f682c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f683153e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c0d3730 con 0x7f682c0d1080 2024-02-18T11:47:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f683153e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682c0cf250 con 0x7f682c161850 2024-02-18T11:47:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f683153e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682c0ce9b0 con 0x7f682c15dc60 2024-02-18T11:47:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f682c15dc60 0x7f682c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682b7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f682c0d1080 0x7f682c0d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50072/0 (socket says 172.21.15.116:50072) 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f682c15dc60 0x7f682c0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34438/0 (socket says 172.21.15.116:34438) 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682b7fe640 1 -- 172.21.15.116:0/341121731 learned_addr learned my addr 172.21.15.116:0/341121731 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682affd640 1 -- 172.21.15.116:0/341121731 >> v1:172.21.15.143:6789/0 conn(0x7f682c161850 legacy=0x7f682c0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682affd640 1 -- 172.21.15.116:0/341121731 >> v1:172.21.15.116:6789/0 conn(0x7f682c0d1080 legacy=0x7f682c0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682affd640 1 -- 172.21.15.116:0/341121731 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0cec90 con 0x7f682c15dc60 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.315+0000 7f682a7fc640 1 -- 172.21.15.116:0/341121731 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711537637 0 0) 0x7f682c0d3730 con 0x7f682c0d1080 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.316+0000 7f682affd640 1 --2- 172.21.15.116:0/341121731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c15dc60 0x7f682c0b6c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f681c0097a0 tx=0x7f681c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5717b935d0a38b8b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.316+0000 7f682a7fc640 1 -- 172.21.15.116:0/341121731 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c030610 con 0x7f682c15dc60 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.316+0000 7f682a7fc640 1 -- 172.21.15.116:0/341121731 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c030770 con 0x7f682c15dc60 2024-02-18T11:47:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.316+0000 7f682a7fc640 1 -- 172.21.15.116:0/341121731 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c030b00 con 0x7f682c15dc60 2024-02-18T11:47:40.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 -- 172.21.15.116:0/341121731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c15dc60 msgr2=0x7f682c0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 --2- 172.21.15.116:0/341121731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c15dc60 0x7f682c0b6c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f681c0097a0 tx=0x7f681c02b960 comp rx=0 tx=0).stop 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 -- 172.21.15.116:0/341121731 shutdown_connections 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 --2- 172.21.15.116:0/341121731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c15dc60 0x7f682c0b6c10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 -- 172.21.15.116:0/341121731 >> 172.21.15.116:0/341121731 conn(0x7f682c0b5910 msgr2=0x7f682c0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 -- 172.21.15.116:0/341121731 shutdown_connections 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.317+0000 7f683153e640 1 -- 172.21.15.116:0/341121731 wait complete. 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.318+0000 7f683153e640 1 Processor -- start 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.318+0000 7f683153e640 1 -- start start 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.318+0000 7f683153e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0d1080 0x7f682c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f682c15dc60 0x7f682c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 0x7f682c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c0cc5b0 con 0x7f682c161850 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682c0ca370 con 0x7f682c0d1080 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682c0cbc80 con 0x7f682c15dc60 2024-02-18T11:47:40.313 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 0x7f682c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 0x7f682c15cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34452/0 (socket says 172.21.15.116:34452) 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 -- 172.21.15.116:0/1368190321 learned_addr learned my addr 172.21.15.116:0/1368190321 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682affd640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0d1080 0x7f682c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f6823fff640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f682c15dc60 0x7f682c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 -- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f682c15dc60 msgr2=0x7f682c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f682c15dc60 0x7f682c15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 -- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0d1080 msgr2=0x7f682c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0d1080 0x7f682c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 -- 172.21.15.116:0/1368190321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c009400 con 0x7f682c161850 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682affd640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0d1080 0x7f682c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f682b7fe640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 0x7f682c15cd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f682400c730 tx=0x7f682400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682400ce40 con 0x7f682c161850 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682400a640 con 0x7f682c161850 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682400aa30 con 0x7f682c161850 2024-02-18T11:47:40.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f682c15e1c0 con 0x7f682c161850 2024-02-18T11:47:40.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.319+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682c0cc240 con 0x7f682c161850 2024-02-18T11:47:40.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.320+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682c067840 con 0x7f682c161850 2024-02-18T11:47:40.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.321+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f682400ab90 con 0x7f682c161850 2024-02-18T11:47:40.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.321+0000 7f68237fe640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f680c06bca0 0x7f680c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.321+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f682408a1f0 con 0x7f682c161850 2024-02-18T11:47:40.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.321+0000 7f682affd640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f680c06bca0 0x7f680c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.316 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.322+0000 7f682affd640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f680c06bca0 0x7f680c06e160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f681c036000 tx=0x7f681c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.318 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.326+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6824054690 con 0x7f682c161850 2024-02-18T11:47:40.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.499+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f682c074520 con 0x7f682c161850 2024-02-18T11:47:40.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.500+0000 7f68237fe640 1 -- 172.21.15.116:0/1368190321 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f6824057d50 con 0x7f682c161850 2024-02-18T11:47:40.493 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:40.493 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f680c06bca0 msgr2=0x7f680c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f680c06bca0 0x7f680c06e160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f681c036000 tx=0x7f681c008030 comp rx=0 tx=0).stop 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 msgr2=0x7f682c15cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 0x7f682c15cd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f682400c730 tx=0x7f682400cbf0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 shutdown_connections 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f680c06bca0 0x7f680c06e160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f682c161850 0x7f682c15cd30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f682c15dc60 0x7f682c15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 --2- 172.21.15.116:0/1368190321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f682c0d1080 0x7f682c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 >> 172.21.15.116:0/1368190321 conn(0x7f682c0b5910 msgr2=0x7f682c0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 shutdown_connections 2024-02-18T11:47:40.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.502+0000 7f683153e640 1 -- 172.21.15.116:0/1368190321 wait complete. 2024-02-18T11:47:40.522 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:47:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.630+0000 7f03ea9c9640 1 Processor -- start 2024-02-18T11:47:40.623 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.630+0000 7f03ea9c9640 1 -- start start 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03ea9c9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03e415dc60 0x7f03e40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03ea9c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e4161850 0x7f03e40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03ea9c9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e40ce780 con 0x7f03e40d0970 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03ea9c9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e40d3730 con 0x7f03e415dc60 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03ea9c9640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cf250 con 0x7f03e4161850 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03e3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03e415dc60 0x7f03e40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03e8f66640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f03e40d0970 0x7f03e40d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50076/0 (socket says 172.21.15.116:50076) 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03e8f66640 1 -- 172.21.15.116:0/944432920 learned_addr learned my addr 172.21.15.116:0/944432920 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.631+0000 7f03e3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03e415dc60 0x7f03e40b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34456/0 (socket says 172.21.15.116:34456) 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e37fe640 1 --2- 172.21.15.116:0/944432920 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e4161850 0x7f03e40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e3fff640 1 -- 172.21.15.116:0/944432920 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e4161850 msgr2=0x7f03e40ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e3fff640 1 --2- 172.21.15.116:0/944432920 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e4161850 0x7f03e40ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e3fff640 1 -- 172.21.15.116:0/944432920 >> v1:172.21.15.116:6789/0 conn(0x7f03e40d0970 legacy=0x7f03e40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e3fff640 1 -- 172.21.15.116:0/944432920 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40ce9b0 con 0x7f03e415dc60 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e2ffd640 1 -- 172.21.15.116:0/944432920 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694322768 0 0) 0x7f03e40ce780 con 0x7f03e40d0970 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.632+0000 7f03e3fff640 1 --2- 172.21.15.116:0/944432920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e40b6c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03d40097a0 tx=0x7f03d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0564f85e3a7f4ed server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03e2ffd640 1 -- 172.21.15.116:0/944432920 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4031830 con 0x7f03e415dc60 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03e2ffd640 1 -- 172.21.15.116:0/944432920 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d4031990 con 0x7f03e415dc60 2024-02-18T11:47:40.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03e2ffd640 1 -- 172.21.15.116:0/944432920 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4031d20 con 0x7f03e415dc60 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 -- 172.21.15.116:0/944432920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 msgr2=0x7f03e40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/944432920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e40b6c10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03d40097a0 tx=0x7f03d402b8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 -- 172.21.15.116:0/944432920 shutdown_connections 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/944432920 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e4161850 0x7f03e40ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/944432920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e40b6c10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 -- 172.21.15.116:0/944432920 >> 172.21.15.116:0/944432920 conn(0x7f03e40b5910 msgr2=0x7f03e40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 -- 172.21.15.116:0/944432920 shutdown_connections 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.633+0000 7f03ea9c9640 1 -- 172.21.15.116:0/944432920 wait complete. 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.634+0000 7f03ea9c9640 1 Processor -- start 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.634+0000 7f03ea9c9640 1 -- start start 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03ea9c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e40d0970 0x7f03e41f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03ea9c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e4146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03ea9c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e4161850 0x7f03e40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03ea9c9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cd960 con 0x7f03e415dc60 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03ea9c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cd0c0 con 0x7f03e4161850 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03ea9c9640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cd3a0 con 0x7f03e40d0970 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e4146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e4146ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34472/0 (socket says 172.21.15.116:34472) 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 -- 172.21.15.116:0/2785254938 learned_addr learned my addr 172.21.15.116:0/2785254938 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e3fff640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e40d0970 0x7f03e41f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e8f66640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e4161850 0x7f03e40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 -- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e40d0970 msgr2=0x7f03e41f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e40d0970 0x7f03e41f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 -- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e4161850 msgr2=0x7f03e40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e4161850 0x7f03e40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e37fe640 1 -- 172.21.15.116:0/2785254938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4009400 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.635+0000 7f03e8f66640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e4161850 0x7f03e40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.636+0000 7f03e37fe640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e4146ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03d800d5f0 tx=0x7f03d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.636+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80125f0 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.636+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d8012750 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.636+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e40cd680 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.636+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8012b00 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.636+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e40ccdd0 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.637+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e4067840 con 0x7f03e415dc60 2024-02-18T11:47:40.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.637+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f03d8012c60 con 0x7f03e415dc60 2024-02-18T11:47:40.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.637+0000 7f03e0ff9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f03d006bca0 0x7f03d006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.638+0000 7f03e3fff640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f03d006bca0 0x7f03d006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.638+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f03d808afb0 con 0x7f03e415dc60 2024-02-18T11:47:40.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.638+0000 7f03e3fff640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f03d006bca0 0x7f03d006e160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03d4036000 tx=0x7f03d403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.642+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03d80be8c0 con 0x7f03e415dc60 2024-02-18T11:47:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.847+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03e4159c90 con 0x7f03e415dc60 2024-02-18T11:47:40.840 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.848+0000 7f03e0ff9640 1 -- 172.21.15.116:0/2785254938 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f03e4159c90 con 0x7f03e415dc60 2024-02-18T11:47:40.840 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:40.840 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:38.907664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-18T11:47:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2024-02-18T11:47:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f03d006bca0 msgr2=0x7f03d006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f03d006bca0 0x7f03d006e160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03d4036000 tx=0x7f03d403c040 comp rx=0 tx=0).stop 2024-02-18T11:47:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 msgr2=0x7f03e4146ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e4146ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03d800d5f0 tx=0x7f03d800dab0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 shutdown_connections 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f03d006bca0 0x7f03d006e160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03e4161850 0x7f03e40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03e415dc60 0x7f03e4146ab0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 --2- 172.21.15.116:0/2785254938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f03e40d0970 0x7f03e41f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 >> 172.21.15.116:0/2785254938 conn(0x7f03e40b5910 msgr2=0x7f03e40c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 shutdown_connections 2024-02-18T11:47:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.850+0000 7f03ea9c9640 1 -- 172.21.15.116:0/2785254938 wait complete. 2024-02-18T11:47:40.853 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.961+0000 7fc5443bc640 1 Processor -- start 2024-02-18T11:47:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.961+0000 7fc5443bc640 1 -- start start 2024-02-18T11:47:40.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc5443bc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc53c0b37d0 0x7fc53c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc5443bc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc53c06da20 0x7fc53c06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc5443bc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53c0b3e10 con 0x7fc53c158260 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc5443bc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c0b5360 con 0x7fc53c06da20 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc5443bc640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c1615c0 con 0x7fc53c0b37d0 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542959640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc53c06da20 0x7fc53c06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542158640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc53c158260 0x7fc53c0b30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50082/0 (socket says 172.21.15.116:50082) 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542158640 1 -- 172.21.15.116:0/1552083356 learned_addr learned my addr 172.21.15.116:0/1552083356 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:40.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc541957640 1 --2- 172.21.15.116:0/1552083356 >> v2:172.21.15.143:3300/0 conn(0x7fc53c0b37d0 0x7fc53c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542959640 1 -- 172.21.15.116:0/1552083356 >> v2:172.21.15.143:3300/0 conn(0x7fc53c0b37d0 msgr2=0x7fc53c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542959640 1 --2- 172.21.15.116:0/1552083356 >> v2:172.21.15.143:3300/0 conn(0x7fc53c0b37d0 0x7fc53c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542959640 1 -- 172.21.15.116:0/1552083356 >> v1:172.21.15.116:6789/0 conn(0x7fc53c158260 legacy=0x7fc53c0b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc542959640 1 -- 172.21.15.116:0/1552083356 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c1618a0 con 0x7fc53c06da20 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.962+0000 7fc541156640 1 -- 172.21.15.116:0/1552083356 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976408135 0 0) 0x7fc53c0b3e10 con 0x7fc53c158260 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.963+0000 7fc542959640 1 --2- 172.21.15.116:0/1552083356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c06da20 0x7fc53c06ada0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc53c072d10 tx=0x7fc53402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfc16aad678cf42 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.963+0000 7fc541156640 1 -- 172.21.15.116:0/1552083356 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53402cc80 con 0x7fc53c06da20 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.963+0000 7fc541156640 1 -- 172.21.15.116:0/1552083356 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53402cde0 con 0x7fc53c06da20 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.963+0000 7fc541156640 1 -- 172.21.15.116:0/1552083356 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534034660 con 0x7fc53c06da20 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.963+0000 7fc5443bc640 1 -- 172.21.15.116:0/1552083356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c06da20 msgr2=0x7fc53c06ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.963+0000 7fc5443bc640 1 --2- 172.21.15.116:0/1552083356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c06da20 0x7fc53c06ada0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc53c072d10 tx=0x7fc53402bed0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.964+0000 7fc5443bc640 1 -- 172.21.15.116:0/1552083356 shutdown_connections 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.964+0000 7fc5443bc640 1 --2- 172.21.15.116:0/1552083356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c06da20 0x7fc53c06ada0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.964+0000 7fc5443bc640 1 --2- 172.21.15.116:0/1552083356 >> v2:172.21.15.143:3300/0 conn(0x7fc53c0b37d0 0x7fc53c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.964+0000 7fc5443bc640 1 -- 172.21.15.116:0/1552083356 >> 172.21.15.116:0/1552083356 conn(0x7fc53c0b23d0 msgr2=0x7fc53c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:40.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.964+0000 7fc5443bc640 1 -- 172.21.15.116:0/1552083356 shutdown_connections 2024-02-18T11:47:40.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.964+0000 7fc5443bc640 1 -- 172.21.15.116:0/1552083356 wait complete. 2024-02-18T11:47:40.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 Processor -- start 2024-02-18T11:47:40.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 -- start start 2024-02-18T11:47:40.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc53c06da20 0x7fc53c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 0x7fc53c0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc53c158260 0x7fc53c1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c154c90 con 0x7fc53c0b37d0 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c153f80 con 0x7fc53c158260 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.965+0000 7fc5443bc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c154640 con 0x7fc53c06da20 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 0x7fc53c0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 0x7fc53c0cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34496/0 (socket says 172.21.15.116:34496) 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 -- 172.21.15.116:0/2087933750 learned_addr learned my addr 172.21.15.116:0/2087933750 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc542158640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc53c06da20 0x7fc53c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc542959640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc53c158260 0x7fc53c1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 -- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc53c06da20 msgr2=0x7fc53c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc53c06da20 0x7fc53c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 -- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc53c158260 msgr2=0x7fc53c1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc53c158260 0x7fc53c1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:40.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 -- 172.21.15.116:0/2087933750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc534009a70 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc542158640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc53c06da20 0x7fc53c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc542959640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc53c158260 0x7fc53c1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc541957640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 0x7fc53c0cc530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00e9b0 tx=0x7fc52c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c009670 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c00c9a0 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c154920 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00cd50 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.966+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c141630 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.967+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53c154e40 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.968+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc52c013470 con 0x7fc53c0b37d0 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.968+0000 7fc532ffd640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc52006bc70 0x7fc52006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:40.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.968+0000 7fc542158640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc52006bc70 0x7fc52006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:40.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.968+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc52c08ad10 con 0x7fc53c0b37d0 2024-02-18T11:47:40.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.968+0000 7fc542158640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc52006bc70 0x7fc52006e130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc538000f60 tx=0x7fc538009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:40.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:40.972+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc53c154e40 con 0x7fc53c0b37d0 2024-02-18T11:47:41.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.143+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc53c074520 con 0x7fc53c0b37d0 2024-02-18T11:47:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.144+0000 7fc532ffd640 1 -- 172.21.15.116:0/2087933750 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fc52c0551b0 con 0x7fc53c0b37d0 2024-02-18T11:47:41.136 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:41.136 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:41.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.145+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc52006bc70 msgr2=0x7fc52006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.145+0000 7fc5443bc640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc52006bc70 0x7fc52006e130 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc538000f60 tx=0x7fc538009040 comp rx=0 tx=0).stop 2024-02-18T11:47:41.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 msgr2=0x7fc53c0cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 0x7fc53c0cc530 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00e9b0 tx=0x7fc52c00ee70 comp rx=0 tx=0).stop 2024-02-18T11:47:41.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 shutdown_connections 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc52006bc70 0x7fc52006e130 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc53c158260 0x7fc53c1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c0b37d0 0x7fc53c0cc530 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 --2- 172.21.15.116:0/2087933750 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc53c06da20 0x7fc53c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 >> 172.21.15.116:0/2087933750 conn(0x7fc53c0b23d0 msgr2=0x7fc53c157850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 shutdown_connections 2024-02-18T11:47:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.146+0000 7fc5443bc640 1 -- 172.21.15.116:0/2087933750 wait complete. 2024-02-18T11:47:41.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-18T11:47:33.859710+0000', 'modified': '2024-02-18T11:47:38.907664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/1897601775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 1897601775}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 1897601775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-18T11:47:41.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-18T11:47:41.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-18T11:47:41.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.662+0000 7f1df5a6f640 1 Processor -- start 2024-02-18T11:47:41.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.662+0000 7f1df5a6f640 1 -- start start 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.662+0000 7f1df5a6f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 0x7f1df00b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.662+0000 7f1df5a6f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df00d1020 con 0x7f1df00b4190 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.662+0000 7f1df5a6f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df00b5190 con 0x7f1df015a870 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.662+0000 7f1df5a6f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00b4600 con 0x7f1df0070970 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1df00b4190 0x7f1df00ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50086/0 (socket says 172.21.15.116:50086) 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deffff640 1 -- 172.21.15.116:0/1080830826 learned_addr learned my addr 172.21.15.116:0/1080830826 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568489234 0 0) 0x7f1df00d1020 con 0x7f1df00b4190 2024-02-18T11:47:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 0x7f1df00b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ddc003520 con 0x7f1df00b4190 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223504778 0 0) 0x7f1ddc003520 con 0x7f1df00b4190 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 msgr2=0x7f1df00b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 0x7f1df00b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 >> v1:172.21.15.143:6789/0 conn(0x7f1df015a870 legacy=0x7f1df0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00d0d40 con 0x7f1df00b4190 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f1de80019f0 con 0x7f1df00b4190 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 >> v1:172.21.15.116:6789/0 conn(0x7f1df00b4190 legacy=0x7f1df00ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ddc0039a0 0x7f1ddc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 0x7f1ddc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ddc004d50 0x7f1ddc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00b4600 con 0x7f1ddc004d50 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc00bc00 con 0x7f1ddc0039a0 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc00bdc0 con 0x7f1df0070970 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1deffff640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ddc004d50 0x7f1ddc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 0x7f1ddc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.663+0000 7f1def7fe640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ddc0039a0 0x7f1ddc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deffff640 1 -- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 msgr2=0x7f1ddc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deffff640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1df0070970 0x7f1ddc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deffff640 1 -- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ddc0039a0 msgr2=0x7f1ddc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deffff640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ddc0039a0 0x7f1ddc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deffff640 1 -- 172.21.15.116:0/1080830826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ddc003520 con 0x7f1ddc004d50 2024-02-18T11:47:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deffff640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ddc004d50 0x7f1ddc0051d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a7f0 tx=0x7f1ddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728bdaffad7fca18 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de80072d0 con 0x7f1ddc004d50 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de802f070 con 0x7f1ddc004d50 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.664+0000 7f1deeffd640 1 -- 172.21.15.116:0/1080830826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de80076b0 con 0x7f1ddc004d50 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1df5a6f640 1 -- 172.21.15.116:0/1080830826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ddc004d50 msgr2=0x7f1ddc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/1080830826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ddc004d50 0x7f1ddc0051d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1de800a7f0 tx=0x7f1ddc0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1deffff640 1 -- 172.21.15.116:0/1080830826 reap_dead start 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1df5a6f640 1 -- 172.21.15.116:0/1080830826 shutdown_connections 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1df5a6f640 1 -- 172.21.15.116:0/1080830826 >> 172.21.15.116:0/1080830826 conn(0x7f1df00b23f0 msgr2=0x7f1df00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1df5a6f640 1 -- 172.21.15.116:0/1080830826 shutdown_connections 2024-02-18T11:47:41.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.665+0000 7f1df5a6f640 1 -- 172.21.15.116:0/1080830826 wait complete. 2024-02-18T11:47:41.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.666+0000 7f1df5a6f640 1 Processor -- start 2024-02-18T11:47:41.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.666+0000 7f1df5a6f640 1 -- start start 2024-02-18T11:47:41.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1df5a6f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1ddc004d50 0x7f1df0140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1df5a6f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df015ac50 0x7f1df0159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1df5a6f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 0x7f1df01408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1df5a6f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0158b00 con 0x7f1df0159ab0 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1df5a6f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df015b0b0 con 0x7f1df015ac50 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1df5a6f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df01584b0 con 0x7f1ddc004d50 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 0x7f1df01408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 0x7f1df01408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34518/0 (socket says 172.21.15.116:34518) 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 -- 172.21.15.116:0/4165215019 learned_addr learned my addr 172.21.15.116:0/4165215019 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df015ac50 0x7f1df0159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1def7fe640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1ddc004d50 0x7f1df0140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 -- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1ddc004d50 msgr2=0x7f1df0140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1ddc004d50 0x7f1df0140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 -- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df015ac50 msgr2=0x7f1df0159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df015ac50 0x7f1df0159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.667+0000 7f1deffff640 1 -- 172.21.15.116:0/4165215019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0158d70 con 0x7f1df0159ab0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df015ac50 0x7f1df0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1deffff640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 0x7f1df01408a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1de8001580 tx=0x7f1df00b5100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1decff9640 1 -- 172.21.15.116:0/4165215019 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8044070 con 0x7f1df0159ab0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1decff9640 1 -- 172.21.15.116:0/4165215019 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8001ae0 con 0x7f1df0159ab0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1decff9640 1 -- 172.21.15.116:0/4165215019 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de80074e0 con 0x7f1df0159ab0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df0159050 con 0x7f1df0159ab0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.668+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df014e5e0 con 0x7f1df0159ab0 2024-02-18T11:47:41.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.669+0000 7f1decff9640 1 -- 172.21.15.116:0/4165215019 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f1de8031030 con 0x7f1df0159ab0 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.670+0000 7f1decff9640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1dd006bc70 0x7f1dd006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.670+0000 7f1def7fe640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1dd006bc70 0x7f1dd006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.670+0000 7f1decff9640 1 -- 172.21.15.116:0/4165215019 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1de80ac440 con 0x7f1df0159ab0 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.670+0000 7f1def7fe640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1dd006bc70 0x7f1dd006e130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1de00035f0 tx=0x7f1de0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.671+0000 7f1df8643740 1 Processor -- start 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.671+0000 7f1df8643740 1 -- start start 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.671+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8930a0 0x55c22a893480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.671+0000 7f1df8643740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 0x55c22a890310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a890850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1df8643740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a823dc0 con 0x55c22a8630c0 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a7d6f20 con 0x55c22a8930a0 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c22a7d4450 con 0x55c22a866c20 2024-02-18T11:47:41.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 0x55c22a890310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 0x55c22a890310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34526/0 (socket says 172.21.15.116:34526) 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 -- 172.21.15.116:0/305037497 learned_addr learned my addr 172.21.15.116:0/305037497 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1deffff640 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a890850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1def7fe640 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8930a0 0x55c22a893480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 -- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 msgr2=0x55c22a890850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a890850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 -- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8930a0 msgr2=0x55c22a893480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8930a0 0x55c22a893480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 -- 172.21.15.116:0/305037497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c22a784f20 con 0x55c22a8630c0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 0x55c22a890310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8004620 tx=0x7f1dd8003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bf1fe2f8942bc0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de57fa640 1 -- 172.21.15.116:0/305037497 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800b970 con 0x55c22a8630c0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de57fa640 1 -- 172.21.15.116:0/305037497 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd800bad0 con 0x55c22a8630c0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.672+0000 7f1de57fa640 1 -- 172.21.15.116:0/305037497 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800be40 con 0x55c22a8630c0 2024-02-18T11:47:41.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 -- 172.21.15.116:0/305037497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 msgr2=0x55c22a890310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 0x55c22a890310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8004620 tx=0x7f1dd8003aa0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 -- 172.21.15.116:0/305037497 shutdown_connections 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a890850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8630c0 0x55c22a890310 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 --2- 172.21.15.116:0/305037497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8930a0 0x55c22a893480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 -- 172.21.15.116:0/305037497 >> 172.21.15.116:0/305037497 conn(0x55c22a865330 msgr2=0x55c22a8adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 -- 172.21.15.116:0/305037497 shutdown_connections 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.673+0000 7f1df8643740 1 -- 172.21.15.116:0/305037497 wait complete. 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 -- start start 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 Processor -- start 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a8a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a88d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a88d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a7cfc30 con 0x55c22a8930a0 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a774d30 con 0x55c22a8c1960 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c22a4a3310 con 0x55c22a866c20 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1df8643740 10 client.0 _ll_register_callbacks cb 0x55c22a77cd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1de6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a88d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1de6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a88d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34532/0 (socket says 172.21.15.116:34532) 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1de6ffd640 1 -- 172.21.15.116:0/3841650089 learned_addr learned my addr 172.21.15.116:0/3841650089 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.674+0000 7f1def7fe640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a8a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1deffff640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a88d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1de6ffd640 1 -- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 msgr2=0x55c22a8a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a8a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1de6ffd640 1 -- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 msgr2=0x55c22a88d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a88d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1de6ffd640 1 -- 172.21.15.116:0/3841650089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd800e040 con 0x55c22a8930a0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a88d140 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8018000 tx=0x7f1dd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916b97d417ce01b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1dce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3841650089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8025070 con 0x55c22a8930a0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3841650089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1dd80207a0 con 0x55c22a8930a0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c22a4cf6c0 con 0x55c22a8930a0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3841650089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dd8020960 con 0x55c22a8930a0 2024-02-18T11:47:41.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 10 client.4428 fetch_fsmap learned FSMap version 6 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55c22a7ce2b0 con 0x55c22a8930a0 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3841650089 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c22a4cf6c0 con 0x55c22a8930a0 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3841650089 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1dd8020ac0 con 0x55c22a8930a0 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 10 client.4428 fetch_fsmap finished waiting for FSMap version 6 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 10 client.4428 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 20 client.4428 populate_metadata read hostname 'smithi116' 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a87b680 0x55c22a87ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 4 client.4428 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.675+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 --> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c22a520270 con 0x55c22a87b680 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1def7fe640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a87b680 0x55c22a87ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1def7fe640 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a87b680 0x55c22a87ba60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46cb91d3a278b0da server_cookie=3196337a31c3056d in_seq=0 out_seq=0 2024-02-18T11:47:41.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1dce7fc640 10 client.4428 ms_handle_connect on v2:172.21.15.116:6834/1897601775 2024-02-18T11:47:41.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3841650089 <== mds.0 v2:172.21.15.116:6834/1897601775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c22a520270 con 0x55c22a87b680 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1dce7fc640 10 client.4428 handle_command_reply: tid=0 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1df8643740 1 client.4428 shutdown 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 msgr2=0x55c22a88d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1df8643740 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a88d140 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8018000 tx=0x7f1dd80046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.676+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 shutdown_connections 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a87b680 0x55c22a87ba60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a88d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1dce7fc640 0 client.4428 ms_handle_reset on v2:172.21.15.116:6834/1897601775 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a88d140 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 --2- 172.21.15.116:0/3841650089 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a866c20 0x55c22a8a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 >> 172.21.15.116:0/3841650089 conn(0x55c22a8630c0 msgr2=0x55c22a8b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 shutdown_connections 2024-02-18T11:47:41.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 1 -- 172.21.15.116:0/3841650089 wait complete. 2024-02-18T11:47:41.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.677+0000 7f1df8643740 20 client.4428 trim_cache size 0 max 16384 2024-02-18T11:47:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.692+0000 7f1df8643740 1 Processor -- start 2024-02-18T11:47:41.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.692+0000 7f1df8643740 1 -- start start 2024-02-18T11:47:41.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a866c20 0x55c22a87ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1df8643740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a8b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8c1960 0x55c22a8b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1df8643740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a755490 con 0x55c22a8930a0 2024-02-18T11:47:41.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a783d60 con 0x55c22a866c20 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c22a87cc50 con 0x55c22a8c1960 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a8b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a8b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34534/0 (socket says 172.21.15.116:34534) 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 -- 172.21.15.116:0/62782675 learned_addr learned my addr 172.21.15.116:0/62782675 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1deffff640 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8c1960 0x55c22a8b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 -- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8c1960 msgr2=0x55c22a8b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8c1960 0x55c22a8b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 -- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a866c20 msgr2=0x55c22a87ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1def7fe640 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a866c20 0x55c22a87ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a866c20 0x55c22a87ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 -- 172.21.15.116:0/62782675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c22a76aa20 con 0x55c22a8930a0 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1def7fe640 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a866c20 0x55c22a87ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.693+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a8b13c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800bfd0 tx=0x7f1dd8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d807bff66994eec server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1de4ff9640 1 -- 172.21.15.116:0/62782675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800b970 con 0x55c22a8930a0 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1de4ff9640 1 -- 172.21.15.116:0/62782675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd8020680 con 0x55c22a8930a0 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1de4ff9640 1 -- 172.21.15.116:0/62782675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8020a10 con 0x55c22a8930a0 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 -- 172.21.15.116:0/62782675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 msgr2=0x55c22a8b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a8b13c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800bfd0 tx=0x7f1dd8025000 comp rx=0 tx=0).stop 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 -- 172.21.15.116:0/62782675 shutdown_connections 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8c1960 0x55c22a8b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a8930a0 0x55c22a8b13c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 --2- 172.21.15.116:0/62782675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a866c20 0x55c22a87ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 -- 172.21.15.116:0/62782675 >> 172.21.15.116:0/62782675 conn(0x55c22a8630c0 msgr2=0x55c22a88d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 -- 172.21.15.116:0/62782675 shutdown_connections 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.694+0000 7f1df8643740 1 -- 172.21.15.116:0/62782675 wait complete. 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.695+0000 7f1df8643740 1 -- start start 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.695+0000 7f1df8643740 1 Processor -- start 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 0x55c22a8c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8930a0 0x55c22a8af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a87ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a8c1460 con 0x55c22a866c20 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c22a8b06b0 con 0x55c22a8c1960 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c22a8bde20 con 0x55c22a8930a0 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 0x55c22a8c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 10 client.0 _ll_register_callbacks cb 0x7f1dd80041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 0x55c22a8c0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34546/0 (socket says 172.21.15.116:34546) 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 -- 172.21.15.116:0/3850795852 learned_addr learned my addr 172.21.15.116:0/3850795852 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8930a0 0x55c22a8af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1deffff640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a87ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 -- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8930a0 msgr2=0x55c22a8af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8930a0 0x55c22a8af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 -- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 msgr2=0x55c22a87ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a87ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 -- 172.21.15.116:0/3850795852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c22a7ce2b0 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1deffff640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a87ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8930a0 0x55c22a8af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1def7fe640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 0x55c22a8c0f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1de00017e0 tx=0x7f1de0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326270a6f01abc5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1dce7fc640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3850795852 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0001d50 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3850795852 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1de001d040 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dd800bad0 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3850795852 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1de0007020 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 10 client.4434 fetch_fsmap learned FSMap version 6 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55c22a790780 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.696+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3850795852 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd800bad0 con 0x55c22a866c20 2024-02-18T11:47:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3850795852 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1de0007180 con 0x55c22a866c20 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 10 client.4434 fetch_fsmap finished waiting for FSMap version 6 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 10 client.4434 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 20 client.4434 populate_metadata read hostname 'smithi116' 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a8affe0 0x55c22a87bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 4 client.4434 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 --> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c22a50c610 con 0x55c22a8affe0 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a8affe0 0x55c22a87bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1dce7fc640 10 client.4434 ms_handle_connect on v2:172.21.15.116:6834/1897601775 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1de6ffd640 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a8affe0 0x55c22a87bfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d65f442ab77442d server_cookie=2e091f64f9b474b7 in_seq=0 out_seq=0 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1dce7fc640 1 -- 172.21.15.116:0/3850795852 <== mds.0 v2:172.21.15.116:6834/1897601775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55c22a50c610 con 0x55c22a8affe0 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1dce7fc640 10 client.4434 handle_command_reply: tid=0 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 client.4434 shutdown 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 msgr2=0x55c22a8c0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 0x55c22a8c0f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1de00017e0 tx=0x7f1de0001190 comp rx=0 tx=0).stop 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 shutdown_connections 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x55c22a8affe0 0x55c22a87bfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c22a8c1960 0x55c22a87ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1dce7fc640 0 client.4434 ms_handle_reset on v2:172.21.15.116:6834/1897601775 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55c22a8930a0 0x55c22a8af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 --2- 172.21.15.116:0/3850795852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c22a866c20 0x55c22a8c0f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 >> 172.21.15.116:0/3850795852 conn(0x55c22a8630c0 msgr2=0x55c22a88f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 shutdown_connections 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 1 -- 172.21.15.116:0/3850795852 wait complete. 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.697+0000 7f1df8643740 20 client.4434 trim_cache size 0 max 16384 2024-02-18T11:47:41.698 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "82e3e917-bb19-48fa-a66a-62df77a0f413", 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4281, 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 7.1797808420000004, 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 6, 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 24, 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 24, 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 13.780993638 2024-02-18T11:47:41.699 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1dd006bc70 msgr2=0x7f1dd006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1dd006bc70 0x7f1dd006e130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1de00035f0 tx=0x7f1de0009040 comp rx=0 tx=0).stop 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 msgr2=0x7f1df01408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 0x7f1df01408a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1de8001580 tx=0x7f1df00b5100 comp rx=0 tx=0).stop 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 shutdown_connections 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1dd006bc70 0x7f1dd006e130 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1df0159ab0 0x7f1df01408a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1df015ac50 0x7f1df0159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 --2- 172.21.15.116:0/4165215019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1ddc004d50 0x7f1df0140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 >> 172.21.15.116:0/4165215019 conn(0x7f1df00b23f0 msgr2=0x7f1df00b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 shutdown_connections 2024-02-18T11:47:41.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.698+0000 7f1df5a6f640 1 -- 172.21.15.116:0/4165215019 wait complete. 2024-02-18T11:47:41.702 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-18T11:47:41.702 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:41.702 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-18T11:47:41.802 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.810+0000 7f8e3a94b640 1 Processor -- start 2024-02-18T11:47:41.803 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.810+0000 7f8e3a94b640 1 -- start start 2024-02-18T11:47:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e3a94b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e3415efb0 0x7f8e340d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e3a94b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e340d0a80 con 0x7f8e34099a50 2024-02-18T11:47:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e3a94b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e340cfe70 con 0x7f8e340cf900 2024-02-18T11:47:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e3a94b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3415f390 con 0x7f8e3415efb0 2024-02-18T11:47:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e33fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e3415efb0 0x7f8e340d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e38ee8640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8e34099a50 0x7f8e340cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50094/0 (socket says 172.21.15.116:50094) 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e38ee8640 1 -- 172.21.15.116:0/247926103 learned_addr learned my addr 172.21.15.116:0/247926103 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e33fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e3415efb0 0x7f8e340d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34550/0 (socket says 172.21.15.116:34550) 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e33fff640 1 -- 172.21.15.116:0/247926103 >> v1:172.21.15.143:6789/0 conn(0x7f8e340cf900 legacy=0x7f8e3406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e33fff640 1 -- 172.21.15.116:0/247926103 >> v1:172.21.15.116:6789/0 conn(0x7f8e34099a50 legacy=0x7f8e340cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e33fff640 1 -- 172.21.15.116:0/247926103 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340d0150 con 0x7f8e3415efb0 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.811+0000 7f8e32ffd640 1 -- 172.21.15.116:0/247926103 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801363901 0 0) 0x7f8e340d0a80 con 0x7f8e34099a50 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e33fff640 1 --2- 172.21.15.116:0/247926103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340d0f10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8e240097a0 tx=0x7f8e2402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93730207f94b111a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e32ffd640 1 -- 172.21.15.116:0/247926103 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2402e070 con 0x7f8e3415efb0 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e32ffd640 1 -- 172.21.15.116:0/247926103 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e240315a0 con 0x7f8e3415efb0 2024-02-18T11:47:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e32ffd640 1 -- 172.21.15.116:0/247926103 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24031930 con 0x7f8e3415efb0 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e3a94b640 1 -- 172.21.15.116:0/247926103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 msgr2=0x7f8e340d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/247926103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340d0f10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8e240097a0 tx=0x7f8e2402b840 comp rx=0 tx=0).stop 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e3a94b640 1 -- 172.21.15.116:0/247926103 shutdown_connections 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/247926103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340d0f10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.812+0000 7f8e3a94b640 1 -- 172.21.15.116:0/247926103 >> 172.21.15.116:0/247926103 conn(0x7f8e340d1790 msgr2=0x7f8e340d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.813+0000 7f8e3a94b640 1 -- 172.21.15.116:0/247926103 shutdown_connections 2024-02-18T11:47:41.809 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.813+0000 7f8e3a94b640 1 -- 172.21.15.116:0/247926103 wait complete. 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 Processor -- start 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 -- start start 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e34099a50 0x7f8e340c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8e340cf900 0x7f8e3414cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3414d090 con 0x7f8e3415efb0 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340b5a30 con 0x7f8e34099a50 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.814+0000 7f8e3a94b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340b63c0 con 0x7f8e340cf900 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34566/0 (socket says 172.21.15.116:34566) 2024-02-18T11:47:41.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 -- 172.21.15.116:0/3492271097 learned_addr learned my addr 172.21.15.116:0/3492271097 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e337fe640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8e340cf900 0x7f8e3414cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e33fff640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e34099a50 0x7f8e340c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 -- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8e340cf900 msgr2=0x7f8e3414cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8e340cf900 0x7f8e3414cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 -- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e34099a50 msgr2=0x7f8e340c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e34099a50 0x7f8e340c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 -- 172.21.15.116:0/3492271097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e24009400 con 0x7f8e3415efb0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e38ee8640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340b6e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00c7a0 tx=0x7f8e2c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00a640 con 0x7f8e3415efb0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00a7a0 con 0x7f8e3415efb0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e340b5c80 con 0x7f8e3415efb0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00abd0 con 0x7f8e3415efb0 2024-02-18T11:47:41.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.815+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e340b6610 con 0x7f8e3415efb0 2024-02-18T11:47:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.816+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e34067840 con 0x7f8e3415efb0 2024-02-18T11:47:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.816+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f8e2c00ad30 con 0x7f8e3415efb0 2024-02-18T11:47:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.817+0000 7f8e30ff9640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e2006bca0 0x7f8e2006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.817+0000 7f8e33fff640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e2006bca0 0x7f8e2006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.817+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8e2c08a1f0 con 0x7f8e3415efb0 2024-02-18T11:47:41.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.817+0000 7f8e33fff640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e2006bca0 0x7f8e2006e160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e24036000 tx=0x7f8e2403d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:41.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:41.821+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e2c054720 con 0x7f8e3415efb0 2024-02-18T11:47:42.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.029+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8e34074520 con 0x7f8e3415efb0 2024-02-18T11:47:42.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.029+0000 7f8e30ff9640 1 -- 172.21.15.116:0/3492271097 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8e2c057de0 con 0x7f8e3415efb0 2024-02-18T11:47:42.022 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e2006bca0 msgr2=0x7f8e2006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e2006bca0 0x7f8e2006e160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e24036000 tx=0x7f8e2403d040 comp rx=0 tx=0).stop 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 msgr2=0x7f8e340b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340b6e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00c7a0 tx=0x7f8e2c00cc60 comp rx=0 tx=0).stop 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 shutdown_connections 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e2006bca0 0x7f8e2006e160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e3415efb0 0x7f8e340b6e10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8e340cf900 0x7f8e3414cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 --2- 172.21.15.116:0/3492271097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e34099a50 0x7f8e340c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 >> 172.21.15.116:0/3492271097 conn(0x7f8e340d1790 msgr2=0x7f8e340caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 shutdown_connections 2024-02-18T11:47:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:42.031+0000 7f8e3a94b640 1 -- 172.21.15.116:0/3492271097 wait complete. 2024-02-18T11:47:42.034 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:47:42.035 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-18T11:47:42.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.035 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-02-18T11:47:42.050 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:47:42.051 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:47:42.052 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:47:42.052 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:47:42.052 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-18T11:47:42.052 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42772sec preferred_lft 42772sec 2024-02-18T11:47:42.052 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-02-18T11:47:42.052 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:47:42.055 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-18T11:47:42.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-18T11:47:42.056 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:47:42.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.222 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:47:42.249 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-02-18T11:47:42.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.251 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-02-18T11:47:42.326 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-02-18T11:47:42.326 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:47:42.327 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-18T11:47:42.327 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-18T11:47:42.327 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:47:42.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.327 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:47:42.327 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:47:42.327 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-18T11:47:42.327 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-18T11:47:42.328 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-18T11:47:42.328 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:47:42.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.571 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-02-18T11:47:42.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.588 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-02-18T11:47:42.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.644 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:47:42.644 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:47:42.645 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-18T11:47:42.645 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-18T11:47:42.645 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:47:42.727 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-18T11:47:42.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.727 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:47:42.727 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:47:42.727 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-18T11:47:42.727 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-18T11:47:42.728 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-18T11:47:42.728 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-18T11:47:42.728 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-18T11:47:42.728 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:47:42.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:42.866 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:47:42.866 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:47:42.866 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-02-18T11:47:42.866 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2024-02-18T11:47:42.866 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:47:42.907 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-18T11:47:42.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:47:42.907 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:42.962 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-18T11:47:42.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:47:42.962 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:43.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-02-18T11:47:43.084 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-02-18T11:47:43.141 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/selinux 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysctl.service 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/nfs/rpc_pipefs 2024-02-18T11:47:43.142 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-18T11:47:43.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:43.143 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-02-18T11:47:43.196 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-18T11:47:43.197 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-18T11:47:43.239 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-02-18T11:47:43.266 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-02-18T11:47:43.321 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/selinux 2024-02-18T11:47:43.322 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysctl.service 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/nfs/rpc_pipefs 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-0 2024-02-18T11:47:43.323 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-1 2024-02-18T11:47:43.324 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-2 2024-02-18T11:47:43.324 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-3 2024-02-18T11:47:43.324 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-02-18T11:47:43.324 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-18T11:47:43.324 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-18T11:47:43.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:43.324 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-02-18T11:47:43.326 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:2024-02-18T11:47:43.333+0000 7fb3db840480 -1 init, newargv = 0x55559070c460 newargc=15 2024-02-18T11:47:43.326 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:ceph-fuse[45005]: starting ceph client 2024-02-18T11:47:43.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:ceph-fuse[45005]: starting fuse 2024-02-18T11:47:43.341 INFO:teuthology.orchestra.run.smithi116.stdout:41 2024-02-18T11:47:43.341 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-02-18T11:47:43.341 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-02-18T11:47:43.341 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-02-18T11:47:43.341 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-02-18T11:47:43.341 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-02-18T11:47:43.341 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-18T11:47:43.342 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.admin")) 2024-02-18T11:47:43.343 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-02-18T11:47:43.461 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.admin.45005.asok 2024-02-18T11:47:43.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-18T11:47:43.471 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.45005.asok 2024-02-18T11:47:43.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:43.472 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.45005.asok status 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "1c920afa8316ea21877f3186850254947531ea22", 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev)", 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "admin", 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "45005", 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-02-18T11:47:43.611 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4452, 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4452 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1440063836 2024-02-18T11:47:43.612 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 1440063836 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4452 192.168.0.1:0/1440063836", 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/1440063836", 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 6, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 24, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-02-18T11:47:43.613 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-02-18T11:47:43.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:43.619 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-02-18T11:47:43.619 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:43.635 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-02-18T11:47:43.635 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:43.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:43.635 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:43.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:43.703 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-18T11:47:43.777 INFO:teuthology.orchestra.run.smithi116.stdout:# file: . 2024-02-18T11:47:43.778 INFO:teuthology.orchestra.run.smithi116.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-18T11:47:43.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:43.778 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:43.778 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:47:43.834 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:43.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.983+0000 7f90f1e8d640 1 Processor -- start 2024-02-18T11:47:43.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.983+0000 7f90f1e8d640 1 -- start start 2024-02-18T11:47:43.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.983+0000 7f90f1e8d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90ec0d0730 0x7f90ec0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.983+0000 7f90f1e8d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec0ccb70 con 0x7f90ec0d0b90 2024-02-18T11:47:43.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.983+0000 7f90f1e8d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec0cf970 con 0x7f90ec0b0750 2024-02-18T11:47:43.978 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.983+0000 7f90f1e8d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ec0cfc50 con 0x7f90ec0d0730 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90ebfff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f90ec0d0b90 0x7f90ec0cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50104/0 (socket says 172.21.15.116:50104) 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90ebfff640 1 -- 172.21.15.116:0/3272329411 learned_addr learned my addr 172.21.15.116:0/3272329411 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90e2ffd640 1 --2- 172.21.15.116:0/3272329411 >> v2:172.21.15.143:3300/0 conn(0x7f90ec0d0730 0x7f90ec0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103820498 0 0) 0x7f90ec0ccb70 con 0x7f90ec0d0b90 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d8003520 con 0x7f90ec0d0b90 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3554846088 0 0) 0x7f90d8003520 con 0x7f90ec0d0b90 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 >> v2:172.21.15.143:3300/0 conn(0x7f90ec0d0730 msgr2=0x7f90ec0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 --2- 172.21.15.116:0/3272329411 >> v2:172.21.15.143:3300/0 conn(0x7f90ec0d0730 0x7f90ec0cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 >> v1:172.21.15.143:6789/0 conn(0x7f90ec0b0750 legacy=0x7f90ec0b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec0cff30 con 0x7f90ec0d0b90 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f90e40019f0 con 0x7f90ec0d0b90 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 >> v1:172.21.15.116:6789/0 conn(0x7f90ec0d0b90 legacy=0x7f90ec0cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d80047c0 0x7f90d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d8005180 0x7f90d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec0cfc50 con 0x7f90d8003970 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d800c020 con 0x7f90d80047c0 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d800c1e0 con 0x7f90d8005180 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90e2ffd640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d80047c0 0x7f90d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90ebfff640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d8005180 0x7f90d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eb7fe640 1 -- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d8005180 msgr2=0x7f90d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d8005180 0x7f90d800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eb7fe640 1 -- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d80047c0 msgr2=0x7f90d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d80047c0 0x7f90d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90eb7fe640 1 -- 172.21.15.116:0/3272329411 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d8003520 con 0x7f90d8003970 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.984+0000 7f90ebfff640 1 -- 172.21.15.116:0/3272329411 reap_dead start 2024-02-18T11:47:43.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.985+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90d8003d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f90dc001750 tx=0x7f90d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d19b11dcf11553d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.985+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0052d0 con 0x7f90d8003970 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.985+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc02e050 con 0x7f90d8003970 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.985+0000 7f90eaffd640 1 -- 172.21.15.116:0/3272329411 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc003830 con 0x7f90d8003970 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 -- 172.21.15.116:0/3272329411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 msgr2=0x7f90d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90d8003d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f90dc001750 tx=0x7f90d80057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 -- 172.21.15.116:0/3272329411 shutdown_connections 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/3272329411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90d8003d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 -- 172.21.15.116:0/3272329411 >> 172.21.15.116:0/3272329411 conn(0x7f90ec0b3380 msgr2=0x7f90ec0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 -- 172.21.15.116:0/3272329411 shutdown_connections 2024-02-18T11:47:43.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.986+0000 7f90f1e8d640 1 -- 172.21.15.116:0/3272329411 wait complete. 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.987+0000 7f90f1e8d640 1 Processor -- start 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.987+0000 7f90f1e8d640 1 -- start start 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90f1e8d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90ec157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90f1e8d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ec155c70 0x7f90ec1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90f1e8d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90ec0c94b0 0x7f90ec0c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90f1e8d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec06c920 con 0x7f90d8003970 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90f1e8d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec1579c0 con 0x7f90ec155c70 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90f1e8d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec1563e0 con 0x7f90ec0c94b0 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90ec157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90ec157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34588/0 (socket says 172.21.15.116:34588) 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 -- 172.21.15.116:0/1453384887 learned_addr learned my addr 172.21.15.116:0/1453384887 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90e2ffd640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ec155c70 0x7f90ec1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.984 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 -- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90ec0c94b0 msgr2=0x7f90ec0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90ebfff640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90ec0c94b0 0x7f90ec0c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90ec0c94b0 0x7f90ec0c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 -- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ec155c70 msgr2=0x7f90ec1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ec155c70 0x7f90ec1f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 -- 172.21.15.116:0/1453384887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec1566c0 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90ebfff640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90ec0c94b0 0x7f90ec0c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90e2ffd640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ec155c70 0x7f90ec1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.988+0000 7f90eb7fe640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90ec157480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005040 tx=0x7f90dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.989+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0076c0 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.989+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc007a30 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.989+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ec1569a0 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.989+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc006be0 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.989+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec156f60 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.990+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ec1570f0 con 0x7f90d8003970 2024-02-18T11:47:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.990+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f90dc006d40 con 0x7f90d8003970 2024-02-18T11:47:43.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.990+0000 7f90e8ff9640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90cc06bca0 0x7f90cc06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:43.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.991+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f90dc0ac7a0 con 0x7f90d8003970 2024-02-18T11:47:43.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.991+0000 7f90e2ffd640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90cc06bca0 0x7f90cc06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:43.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.991+0000 7f90e2ffd640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90cc06bca0 0x7f90cc06e160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0685c0 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:43.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:43.995+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90dc076bb0 con 0x7f90d8003970 2024-02-18T11:47:44.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.168+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90ec074520 con 0x7f90d8003970 2024-02-18T11:47:44.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.169+0000 7f90e8ff9640 1 -- 172.21.15.116:0/1453384887 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f90dc041090 con 0x7f90d8003970 2024-02-18T11:47:44.161 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:44.161 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90cc06bca0 msgr2=0x7f90cc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90cc06bca0 0x7f90cc06e160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0685c0 tx=0x7f90d4008040 comp rx=0 tx=0).stop 2024-02-18T11:47:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 msgr2=0x7f90ec157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90ec157480 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005040 tx=0x7f90dc001120 comp rx=0 tx=0).stop 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 shutdown_connections 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90cc06bca0 0x7f90cc06e160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90ec0c94b0 0x7f90ec0c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ec155c70 0x7f90ec1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 --2- 172.21.15.116:0/1453384887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90d8003970 0x7f90ec157480 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 >> 172.21.15.116:0/1453384887 conn(0x7f90ec0b3380 msgr2=0x7f90ec0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 shutdown_connections 2024-02-18T11:47:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.171+0000 7f90f1e8d640 1 -- 172.21.15.116:0/1453384887 wait complete. 2024-02-18T11:47:44.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:47:44.175 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:47:44.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.288+0000 7fa086c55640 1 Processor -- start 2024-02-18T11:47:44.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.288+0000 7fa086c55640 1 -- start start 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa086c55640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 0x7fa08006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa086c55640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0800b2a50 con 0x7fa080144090 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa086c55640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0800b35e0 con 0x7fa08006e870 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa086c55640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08006ed00 con 0x7fa0800b31a0 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa0849f1640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa080144090 0x7fa0800cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50116/0 (socket says 172.21.15.116:50116) 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa0849f1640 1 -- 172.21.15.116:0/2052273327 learned_addr learned my addr 172.21.15.116:0/2052273327 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07ffff640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 0x7fa08006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849816909 0 0) 0x7fa0800b2a50 con 0x7fa080144090 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa070003520 con 0x7fa080144090 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223362753 0 0) 0x7fa0800b35e0 con 0x7fa08006e870 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0800b2a50 con 0x7fa08006e870 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813779614 0 0) 0x7fa070003520 con 0x7fa080144090 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 msgr2=0x7fa08006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 0x7fa08006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 >> v1:172.21.15.143:6789/0 conn(0x7fa08006e870 legacy=0x7fa08015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800cd0f0 con 0x7fa080144090 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fa074001670 con 0x7fa080144090 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 >> v1:172.21.15.116:6789/0 conn(0x7fa080144090 legacy=0x7fa0800cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 0x7fa070003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa070004090 0x7fa0700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa070004d00 0x7fa070005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08006ed00 con 0x7fa070004d00 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07000bb10 con 0x7fa070004090 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.289+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07000bca0 con 0x7fa0800b31a0 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0851f2640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa070004d00 0x7fa070005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa07ffff640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa070004090 0x7fa0700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0849f1640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 0x7fa070003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0851f2640 1 -- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 msgr2=0x7fa070003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0851f2640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa0800b31a0 0x7fa070003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0851f2640 1 -- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa070004090 msgr2=0x7fa0700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0851f2640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa070004090 0x7fa0700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.290+0000 7fa0851f2640 1 -- 172.21.15.116:0/2052273327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070003520 con 0x7fa070004d00 2024-02-18T11:47:44.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa0851f2640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa070004d00 0x7fa070005180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa078001fd0 tx=0x7fa0700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb31d2001ae0c972 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078003b70 con 0x7fa070004d00 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa078003cd0 con 0x7fa070004d00 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa07f7fe640 1 -- 172.21.15.116:0/2052273327 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0780052f0 con 0x7fa070004d00 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa086c55640 1 -- 172.21.15.116:0/2052273327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa070004d00 msgr2=0x7fa070005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa086c55640 1 --2- 172.21.15.116:0/2052273327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa070004d00 0x7fa070005180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa078001fd0 tx=0x7fa0700057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa0851f2640 1 -- 172.21.15.116:0/2052273327 reap_dead start 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa086c55640 1 -- 172.21.15.116:0/2052273327 shutdown_connections 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa086c55640 1 -- 172.21.15.116:0/2052273327 >> 172.21.15.116:0/2052273327 conn(0x7fa080067000 msgr2=0x7fa08006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:44.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa086c55640 1 -- 172.21.15.116:0/2052273327 shutdown_connections 2024-02-18T11:47:44.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.291+0000 7fa086c55640 1 -- 172.21.15.116:0/2052273327 wait complete. 2024-02-18T11:47:44.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.292+0000 7fa086c55640 1 Processor -- start 2024-02-18T11:47:44.289 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 -- start start 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa070004d00 0x7fa08013e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 0x7fa0800afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0800aab60 0x7fa0800aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800ab520 con 0x7fa0800af6e0 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800b0e60 con 0x7fa0800aab60 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa086c55640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08013f020 con 0x7fa070004d00 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 0x7fa0800afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa07ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 0x7fa0800afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34608/0 (socket says 172.21.15.116:34608) 2024-02-18T11:47:44.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.293+0000 7fa07ffff640 1 -- 172.21.15.116:0/4083726360 learned_addr learned my addr 172.21.15.116:0/4083726360 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa0849f1640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa070004d00 0x7fa08013e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa0851f2640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0800aab60 0x7fa0800aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07ffff640 1 -- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa070004d00 msgr2=0x7fa08013e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07ffff640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa070004d00 0x7fa08013e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07ffff640 1 -- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0800aab60 msgr2=0x7fa0800aafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07ffff640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0800aab60 0x7fa0800aafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07ffff640 1 -- 172.21.15.116:0/4083726360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800b0250 con 0x7fa0800af6e0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa0851f2640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0800aab60 0x7fa0800aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07ffff640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 0x7fa0800afb60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0119a0 tx=0x7fa06c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c00faf0 con 0x7fa0800af6e0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c00fc50 con 0x7fa0800af6e0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0800b0530 con 0x7fa0800af6e0 2024-02-18T11:47:44.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c014440 con 0x7fa0800af6e0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.294+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0800b0af0 con 0x7fa0800af6e0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.295+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0800cbe40 con 0x7fa0800af6e0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.296+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa06c0145a0 con 0x7fa0800af6e0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.296+0000 7fa07d7fa640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa06406bc70 0x7fa06406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.296+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa06c08b380 con 0x7fa0800af6e0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.296+0000 7fa0849f1640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa06406bc70 0x7fa06406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.296+0000 7fa0849f1640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa06406bc70 0x7fa06406e130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0800aa850 tx=0x7fa074009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.300+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa06c055820 con 0x7fa0800af6e0 2024-02-18T11:47:44.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.472+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa080074520 con 0x7fa0800af6e0 2024-02-18T11:47:44.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.472+0000 7fa07d7fa640 1 -- 172.21.15.116:0/4083726360 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa06c058ee0 con 0x7fa0800af6e0 2024-02-18T11:47:44.464 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:47:44.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa06406bc70 msgr2=0x7fa06406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa06406bc70 0x7fa06406e130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0800aa850 tx=0x7fa074009040 comp rx=0 tx=0).stop 2024-02-18T11:47:44.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 msgr2=0x7fa0800afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 0x7fa0800afb60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0119a0 tx=0x7fa06c011e60 comp rx=0 tx=0).stop 2024-02-18T11:47:44.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 shutdown_connections 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa06406bc70 0x7fa06406e130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0800aab60 0x7fa0800aafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0800af6e0 0x7fa0800afb60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 --2- 172.21.15.116:0/4083726360 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa070004d00 0x7fa08013e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 >> 172.21.15.116:0/4083726360 conn(0x7fa080067000 msgr2=0x7fa08015bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 shutdown_connections 2024-02-18T11:47:44.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.474+0000 7fa086c55640 1 -- 172.21.15.116:0/4083726360 wait complete. 2024-02-18T11:47:44.477 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:47:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.589+0000 7f809414f640 1 Processor -- start 2024-02-18T11:47:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.590+0000 7f809414f640 1 -- start start 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.590+0000 7f809414f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f808c0711c0 0x7f808c071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.590+0000 7f809414f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c15da30 0x7f808c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.590+0000 7f809414f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c15c960 con 0x7f808c145230 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.590+0000 7f809414f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808c15d050 con 0x7f808c0711c0 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.590+0000 7f809414f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c145610 con 0x7f808c15da30 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f80916ea640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f808c0711c0 0x7f808c071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f8091eeb640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f808c145230 0x7f808c15be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50122/0 (socket says 172.21.15.116:50122) 2024-02-18T11:47:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f8091eeb640 1 -- 172.21.15.116:0/1750190765 learned_addr learned my addr 172.21.15.116:0/1750190765 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f80926ec640 1 --2- 172.21.15.116:0/1750190765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c15da30 0x7f808c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f80916ea640 1 -- 172.21.15.116:0/1750190765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c15da30 msgr2=0x7f808c15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f80916ea640 1 --2- 172.21.15.116:0/1750190765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c15da30 0x7f808c15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f80916ea640 1 -- 172.21.15.116:0/1750190765 >> v1:172.21.15.116:6789/0 conn(0x7f808c145230 legacy=0x7f808c15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f8090ee9640 1 -- 172.21.15.116:0/1750190765 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547476155 0 0) 0x7f808c15c960 con 0x7f808c145230 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.591+0000 7f80916ea640 1 -- 172.21.15.116:0/1750190765 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c15cc40 con 0x7f808c0711c0 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f80916ea640 1 --2- 172.21.15.116:0/1750190765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c0711c0 0x7f808c071650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f80800097a0 tx=0x7f808002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee417ed463033bd server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f8090ee9640 1 -- 172.21.15.116:0/1750190765 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080031830 con 0x7f808c0711c0 2024-02-18T11:47:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f8090ee9640 1 -- 172.21.15.116:0/1750190765 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080031990 con 0x7f808c0711c0 2024-02-18T11:47:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f8090ee9640 1 -- 172.21.15.116:0/1750190765 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080031d20 con 0x7f808c0711c0 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 -- 172.21.15.116:0/1750190765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c0711c0 msgr2=0x7f808c071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 --2- 172.21.15.116:0/1750190765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c0711c0 0x7f808c071650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f80800097a0 tx=0x7f808002b8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 -- 172.21.15.116:0/1750190765 shutdown_connections 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 --2- 172.21.15.116:0/1750190765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c15da30 0x7f808c15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 --2- 172.21.15.116:0/1750190765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c0711c0 0x7f808c071650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 -- 172.21.15.116:0/1750190765 >> 172.21.15.116:0/1750190765 conn(0x7f808c067000 msgr2=0x7f808c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 -- 172.21.15.116:0/1750190765 shutdown_connections 2024-02-18T11:47:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.592+0000 7f809414f640 1 -- 172.21.15.116:0/1750190765 wait complete. 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 Processor -- start 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 -- start start 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c0711c0 0x7f808c0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 0x7f808c13f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808c15da30 0x7f808c1f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b4600 con 0x7f808c145230 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b26f0 con 0x7f808c15da30 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f809414f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c13fdc0 con 0x7f808c0711c0 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f80916ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 0x7f808c13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f80916ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 0x7f808c13f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34616/0 (socket says 172.21.15.116:34616) 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f80916ea640 1 -- 172.21.15.116:0/670277275 learned_addr learned my addr 172.21.15.116:0/670277275 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.594+0000 7f8091eeb640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c0711c0 0x7f808c0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80926ec640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808c15da30 0x7f808c1f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80916ea640 1 -- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c0711c0 msgr2=0x7f808c0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80916ea640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c0711c0 0x7f808c0b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80916ea640 1 -- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808c15da30 msgr2=0x7f808c1f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80916ea640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808c15da30 0x7f808c1f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80916ea640 1 -- 172.21.15.116:0/670277275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009400 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f80916ea640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 0x7f808c13f880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f808002bee0 tx=0x7f80800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808003a6f0 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808003a870 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c1f4770 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808003ac20 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.595+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c0b2c20 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.596+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c071650 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.597+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f808004c3b0 con 0x7f808c145230 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.597+0000 7f807e7fc640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f807006bc70 0x7f807006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.597+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f80800ac640 con 0x7f808c145230 2024-02-18T11:47:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.597+0000 7f8091eeb640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f807006bc70 0x7f807006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.597+0000 7f8091eeb640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f807006bc70 0x7f807006e130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8088000f60 tx=0x7f8088009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.601+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8080076c20 con 0x7f808c145230 2024-02-18T11:47:44.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.771+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f808c074520 con 0x7f808c145230 2024-02-18T11:47:44.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.771+0000 7f807e7fc640 1 -- 172.21.15.116:0/670277275 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f808007a2e0 con 0x7f808c145230 2024-02-18T11:47:44.764 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:47:44.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f807006bc70 msgr2=0x7f807006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f807006bc70 0x7f807006e130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8088000f60 tx=0x7f8088009040 comp rx=0 tx=0).stop 2024-02-18T11:47:44.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 msgr2=0x7f808c13f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 0x7f808c13f880 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f808002bee0 tx=0x7f80800046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 shutdown_connections 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f807006bc70 0x7f807006e130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f808c15da30 0x7f808c1f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f808c145230 0x7f808c13f880 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 --2- 172.21.15.116:0/670277275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f808c0711c0 0x7f808c0b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 >> 172.21.15.116:0/670277275 conn(0x7f808c067000 msgr2=0x7f808c0cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 shutdown_connections 2024-02-18T11:47:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.773+0000 7f809414f640 1 -- 172.21.15.116:0/670277275 wait complete. 2024-02-18T11:47:44.776 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:47:44.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.887+0000 7f12ea74f640 1 Processor -- start 2024-02-18T11:47:44.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.887+0000 7f12ea74f640 1 -- start start 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12ea74f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12e4150ed0 0x7f12e40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12ea74f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f12e40cdd20 0x7f12e4071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12ea74f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e415a480 con 0x7f12e40b4740 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12ea74f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e415a760 con 0x7f12e40cdd20 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12ea74f640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e415aa40 con 0x7f12e4150ed0 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12e8cec640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f12e40b4740 0x7f12e406d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50124/0 (socket says 172.21.15.116:50124) 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12e37fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f12e40cdd20 0x7f12e4071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.888+0000 7f12e8cec640 1 -- 172.21.15.116:0/2147450622 learned_addr learned my addr 172.21.15.116:0/2147450622 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e3fff640 1 --2- 172.21.15.116:0/2147450622 >> v2:172.21.15.143:3300/0 conn(0x7f12e4150ed0 0x7f12e40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e37fe640 1 -- 172.21.15.116:0/2147450622 >> v2:172.21.15.143:3300/0 conn(0x7f12e4150ed0 msgr2=0x7f12e40b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e37fe640 1 --2- 172.21.15.116:0/2147450622 >> v2:172.21.15.143:3300/0 conn(0x7f12e4150ed0 0x7f12e40b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e37fe640 1 -- 172.21.15.116:0/2147450622 >> v1:172.21.15.116:6789/0 conn(0x7f12e40b4740 legacy=0x7f12e406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e2ffd640 1 -- 172.21.15.116:0/2147450622 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369050150 0 0) 0x7f12e415a480 con 0x7f12e40b4740 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e37fe640 1 -- 172.21.15.116:0/2147450622 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e415ad20 con 0x7f12e40cdd20 2024-02-18T11:47:44.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e3fff640 1 --2- 172.21.15.116:0/2147450622 >> v2:172.21.15.143:3300/0 conn(0x7f12e4150ed0 0x7f12e40b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:44.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e37fe640 1 --2- 172.21.15.116:0/2147450622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e40cdd20 0x7f12e4071a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f12d80097a0 tx=0x7f12d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a47cc3c56445b99 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.889+0000 7f12e2ffd640 1 -- 172.21.15.116:0/2147450622 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d802e070 con 0x7f12e40cdd20 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12e2ffd640 1 -- 172.21.15.116:0/2147450622 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80315a0 con 0x7f12e40cdd20 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12e2ffd640 1 -- 172.21.15.116:0/2147450622 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8031930 con 0x7f12e40cdd20 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 -- 172.21.15.116:0/2147450622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e40cdd20 msgr2=0x7f12e4071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 --2- 172.21.15.116:0/2147450622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e40cdd20 0x7f12e4071a50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f12d80097a0 tx=0x7f12d802b840 comp rx=0 tx=0).stop 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 -- 172.21.15.116:0/2147450622 shutdown_connections 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 --2- 172.21.15.116:0/2147450622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e40cdd20 0x7f12e4071a50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 --2- 172.21.15.116:0/2147450622 >> v2:172.21.15.143:3300/0 conn(0x7f12e4150ed0 0x7f12e40b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 -- 172.21.15.116:0/2147450622 >> 172.21.15.116:0/2147450622 conn(0x7f12e4067000 msgr2=0x7f12e406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 -- 172.21.15.116:0/2147450622 shutdown_connections 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.890+0000 7f12ea74f640 1 -- 172.21.15.116:0/2147450622 wait complete. 2024-02-18T11:47:44.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.891+0000 7f12ea74f640 1 Processor -- start 2024-02-18T11:47:44.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.891+0000 7f12ea74f640 1 -- start start 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12ea74f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12e40b4740 0x7f12e41453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12ea74f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12e40cdd20 0x7f12e413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12ea74f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 0x7f12e4141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12ea74f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4141100 con 0x7f12e4150ed0 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12ea74f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4140500 con 0x7f12e40cdd20 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12ea74f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4141a60 con 0x7f12e40b4740 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 0x7f12e4141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 0x7f12e4141520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34630/0 (socket says 172.21.15.116:34630) 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 -- 172.21.15.116:0/1766468448 learned_addr learned my addr 172.21.15.116:0/1766468448 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:44.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e37fe640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12e40cdd20 0x7f12e413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e3fff640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12e40b4740 0x7f12e41453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 -- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12e40b4740 msgr2=0x7f12e41453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12e40b4740 0x7f12e41453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 -- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12e40cdd20 msgr2=0x7f12e413ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12e40cdd20 0x7f12e413ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.892+0000 7f12e8cec640 1 -- 172.21.15.116:0/1766468448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8009400 con 0x7f12e4150ed0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.893+0000 7f12e8cec640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 0x7f12e4141520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00c7b0 tx=0x7f12dc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.893+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00a5e0 con 0x7f12e4150ed0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.893+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc00a740 con 0x7f12e4150ed0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.893+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4145a50 con 0x7f12e4150ed0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.893+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00ab30 con 0x7f12e4150ed0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.893+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4140a40 con 0x7f12e4150ed0 2024-02-18T11:47:44.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.894+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4141240 con 0x7f12e4150ed0 2024-02-18T11:47:44.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.894+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f12dc00ac90 con 0x7f12e4150ed0 2024-02-18T11:47:44.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.895+0000 7f12e0ff9640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f12d006bcc0 0x7f12d006e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:44.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.895+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f12dc08a250 con 0x7f12e4150ed0 2024-02-18T11:47:44.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.895+0000 7f12e3fff640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f12d006bcc0 0x7f12d006e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:44.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.895+0000 7f12e3fff640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f12d006bcc0 0x7f12d006e180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12e4145ca0 tx=0x7f12d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:44.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:44.899+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12e4141240 con 0x7f12e4150ed0 2024-02-18T11:47:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.069+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12e4141240 con 0x7f12e4150ed0 2024-02-18T11:47:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.070+0000 7f12e0ff9640 1 -- 172.21.15.116:0/1766468448 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12e4141240 con 0x7f12e4150ed0 2024-02-18T11:47:45.062 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:47:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.071+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f12d006bcc0 msgr2=0x7f12d006e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.071+0000 7f12ea74f640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f12d006bcc0 0x7f12d006e180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12e4145ca0 tx=0x7f12d4008040 comp rx=0 tx=0).stop 2024-02-18T11:47:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.071+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 msgr2=0x7f12e4141520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.071+0000 7f12ea74f640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 0x7f12e4141520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00c7b0 tx=0x7f12dc00cc70 comp rx=0 tx=0).stop 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 shutdown_connections 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f12d006bcc0 0x7f12d006e180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12e4150ed0 0x7f12e4141520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12e40cdd20 0x7f12e413ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 --2- 172.21.15.116:0/1766468448 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12e40b4740 0x7f12e41453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 >> 172.21.15.116:0/1766468448 conn(0x7f12e4067000 msgr2=0x7f12e4148970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 shutdown_connections 2024-02-18T11:47:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.072+0000 7f12ea74f640 1 -- 172.21.15.116:0/1766468448 wait complete. 2024-02-18T11:47:45.074 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:47:45.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.185+0000 7fb030aff640 1 Processor -- start 2024-02-18T11:47:45.177 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.185+0000 7fb030aff640 1 -- start start 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb030aff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb02c15f8c0 0x7fb02c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb030aff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c15fca0 0x7fb02c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb030aff640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02c0d06c0 con 0x7fb02c0d0220 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb030aff640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c099bf0 con 0x7fb02c15f8c0 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb030aff640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c099ed0 con 0x7fb02c15fca0 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb02c15f8c0 0x7fb02c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb02c15f8c0 0x7fb02c09a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34636/0 (socket says 172.21.15.116:34636) 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 -- 172.21.15.116:0/4008584462 learned_addr learned my addr 172.21.15.116:0/4008584462 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb023fff640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c15fca0 0x7fb02c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 -- 172.21.15.116:0/4008584462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c15fca0 msgr2=0x7fb02c0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c15fca0 0x7fb02c0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 -- 172.21.15.116:0/4008584462 >> v1:172.21.15.143:6789/0 conn(0x7fb02c0d0220 legacy=0x7fb02c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb02a59c640 1 -- 172.21.15.116:0/4008584462 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c09a1b0 con 0x7fb02c15f8c0 2024-02-18T11:47:45.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.186+0000 7fb023fff640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c15fca0 0x7fb02c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.187+0000 7fb02a59c640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15f8c0 0x7fb02c09a9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009870 tx=0x7fb01c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7784d02d847c3e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.187+0000 7fb029d9b640 1 -- 172.21.15.116:0/4008584462 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c02f840 con 0x7fb02c15f8c0 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.187+0000 7fb029d9b640 1 -- 172.21.15.116:0/4008584462 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c02f9a0 con 0x7fb02c15f8c0 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.187+0000 7fb029d9b640 1 -- 172.21.15.116:0/4008584462 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c02fd30 con 0x7fb02c15f8c0 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 -- 172.21.15.116:0/4008584462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15f8c0 msgr2=0x7fb02c09a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15f8c0 0x7fb02c09a9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb01c009870 tx=0x7fb01c02ba20 comp rx=0 tx=0).stop 2024-02-18T11:47:45.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 -- 172.21.15.116:0/4008584462 shutdown_connections 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c15fca0 0x7fb02c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 --2- 172.21.15.116:0/4008584462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15f8c0 0x7fb02c09a9b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 -- 172.21.15.116:0/4008584462 >> 172.21.15.116:0/4008584462 conn(0x7fb02c0b31f0 msgr2=0x7fb02c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 -- 172.21.15.116:0/4008584462 shutdown_connections 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.188+0000 7fb030aff640 1 -- 172.21.15.116:0/4008584462 wait complete. 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.189+0000 7fb030aff640 1 Processor -- start 2024-02-18T11:47:45.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.189+0000 7fb030aff640 1 -- start start 2024-02-18T11:47:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c0d0220 0x7fb02c15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb02c15f8c0 0x7fb02c0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 0x7fb02c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0b5840 con 0x7fb02c15fca0 2024-02-18T11:47:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0b4bc0 con 0x7fb02c15f8c0 2024-02-18T11:47:45.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0b4ea0 con 0x7fb02c0d0220 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 0x7fb02c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 0x7fb02c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34642/0 (socket says 172.21.15.116:34642) 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 -- 172.21.15.116:0/2580814535 learned_addr learned my addr 172.21.15.116:0/2580814535 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02a59c640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c0d0220 0x7fb02c15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb023fff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb02c15f8c0 0x7fb02c0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 -- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c0d0220 msgr2=0x7fb02c15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c0d0220 0x7fb02c15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 -- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb02c15f8c0 msgr2=0x7fb02c0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb02c15f8c0 0x7fb02c0b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 -- 172.21.15.116:0/2580814535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0094d0 con 0x7fb02c15fca0 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb02ad9d640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 0x7fb02c0b5c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb02400c7a0 tx=0x7fb02400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024003b70 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024003cf0 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c0b5180 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024003ea0 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.190+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c15c860 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.192+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c067840 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.192+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fb02400aa90 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.192+0000 7fb022ffd640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb00c06bc70 0x7fb00c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.192+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb02408a6c0 con 0x7fb02c15fca0 2024-02-18T11:47:45.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.192+0000 7fb02a59c640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb00c06bc70 0x7fb00c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.193+0000 7fb02a59c640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb00c06bc70 0x7fb00c06e130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb01c036000 tx=0x7fb01c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.196+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb024054bf0 con 0x7fb02c15fca0 2024-02-18T11:47:45.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.367+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb02c0d0ed0 con 0x7fb02c15fca0 2024-02-18T11:47:45.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.367+0000 7fb022ffd640 1 -- 172.21.15.116:0/2580814535 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb0240582b0 con 0x7fb02c15fca0 2024-02-18T11:47:45.361 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.369+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb00c06bc70 msgr2=0x7fb00c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.369+0000 7fb030aff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb00c06bc70 0x7fb00c06e130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb01c036000 tx=0x7fb01c008030 comp rx=0 tx=0).stop 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.369+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 msgr2=0x7fb02c0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.369+0000 7fb030aff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 0x7fb02c0b5c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb02400c7a0 tx=0x7fb02400cc60 comp rx=0 tx=0).stop 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 shutdown_connections 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb00c06bc70 0x7fb00c06e130 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb02c15fca0 0x7fb02c0b5c60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb02c15f8c0 0x7fb02c0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 --2- 172.21.15.116:0/2580814535 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb02c0d0220 0x7fb02c15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 >> 172.21.15.116:0/2580814535 conn(0x7fb02c0b31f0 msgr2=0x7fb02c0c6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 shutdown_connections 2024-02-18T11:47:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.370+0000 7fb030aff640 1 -- 172.21.15.116:0/2580814535 wait complete. 2024-02-18T11:47:45.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:47:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.481+0000 7f9c54593640 1 Processor -- start 2024-02-18T11:47:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.481+0000 7f9c54593640 1 -- start start 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.482+0000 7f9c54593640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c4c15db40 0x7f9c4c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.482+0000 7f9c54593640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b5880 0x7f9c4c06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.482+0000 7f9c54593640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c070f50 con 0x7f9c4c0b4bc0 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.482+0000 7f9c54593640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c071230 con 0x7f9c4c15db40 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.482+0000 7f9c54593640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c071510 con 0x7f9c4c0b5880 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c5232f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c4c15db40 0x7f9c4c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c51b2e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9c4c0b4bc0 0x7f9c4c0b5170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50134/0 (socket says 172.21.15.116:50134) 2024-02-18T11:47:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c51b2e640 1 -- 172.21.15.116:0/2794627533 learned_addr learned my addr 172.21.15.116:0/2794627533 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c52b30640 1 --2- 172.21.15.116:0/2794627533 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b5880 0x7f9c4c06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c5232f640 1 -- 172.21.15.116:0/2794627533 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b5880 msgr2=0x7f9c4c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c5232f640 1 --2- 172.21.15.116:0/2794627533 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b5880 0x7f9c4c06bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c5232f640 1 -- 172.21.15.116:0/2794627533 >> v1:172.21.15.116:6789/0 conn(0x7f9c4c0b4bc0 legacy=0x7f9c4c0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c5132d640 1 -- 172.21.15.116:0/2794627533 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248419377 0 0) 0x7f9c4c070f50 con 0x7f9c4c0b4bc0 2024-02-18T11:47:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.483+0000 7f9c5232f640 1 -- 172.21.15.116:0/2794627533 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0717f0 con 0x7f9c4c15db40 2024-02-18T11:47:45.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c5232f640 1 --2- 172.21.15.116:0/2794627533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c0b4680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0097a0 tx=0x7f9c3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f1c5294fc8fbb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c5132d640 1 -- 172.21.15.116:0/2794627533 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c031830 con 0x7f9c4c15db40 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c5132d640 1 -- 172.21.15.116:0/2794627533 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c031990 con 0x7f9c4c15db40 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c5132d640 1 -- 172.21.15.116:0/2794627533 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c031d20 con 0x7f9c4c15db40 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 -- 172.21.15.116:0/2794627533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 msgr2=0x7f9c4c0b4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 --2- 172.21.15.116:0/2794627533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c0b4680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0097a0 tx=0x7f9c3c02b8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 -- 172.21.15.116:0/2794627533 shutdown_connections 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 --2- 172.21.15.116:0/2794627533 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b5880 0x7f9c4c06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 --2- 172.21.15.116:0/2794627533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c0b4680 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 -- 172.21.15.116:0/2794627533 >> 172.21.15.116:0/2794627533 conn(0x7f9c4c06f460 msgr2=0x7f9c4c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 -- 172.21.15.116:0/2794627533 shutdown_connections 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.484+0000 7f9c54593640 1 -- 172.21.15.116:0/2794627533 wait complete. 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.485+0000 7f9c54593640 1 Processor -- start 2024-02-18T11:47:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.485+0000 7f9c54593640 1 -- start start 2024-02-18T11:47:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c54593640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b4bc0 0x7f9c4c14bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c54593640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c4c0b5880 0x7f9c4c15f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c54593640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c54593640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c14fdf0 con 0x7f9c4c15db40 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c54593640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c15fae0 con 0x7f9c4c0b5880 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c54593640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c14cce0 con 0x7f9c4c0b4bc0 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c52b30640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c52b30640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c15f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34666/0 (socket says 172.21.15.116:34666) 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c52b30640 1 -- 172.21.15.116:0/1952280271 learned_addr learned my addr 172.21.15.116:0/1952280271 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.486+0000 7f9c5232f640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b4bc0 0x7f9c4c14bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c52b30640 1 -- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b4bc0 msgr2=0x7f9c4c14bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c52b30640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b4bc0 0x7f9c4c14bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c52b30640 1 -- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c4c0b5880 msgr2=0x7f9c4c15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c52b30640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c4c0b5880 0x7f9c4c15f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c52b30640 1 -- 172.21.15.116:0/1952280271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009400 con 0x7f9c4c15db40 2024-02-18T11:47:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c5232f640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b4bc0 0x7f9c4c14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c52b30640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c15f5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400efc0 tx=0x7f9c4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44009ca0 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4400f040 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c14cfc0 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c440146c0 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.487+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c14d580 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.488+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c0b6170 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.488+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9c44014820 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.489+0000 7f9c42ffd640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c3006bc70 0x7f9c3006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.489+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9c4408b9f0 con 0x7f9c4c15db40 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.489+0000 7f9c5232f640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c3006bc70 0x7f9c3006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.489+0000 7f9c5232f640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c3006bc70 0x7f9c3006e130 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c036000 tx=0x7f9c3c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.493+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c44055f20 con 0x7f9c4c15db40 2024-02-18T11:47:45.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.677+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9c4c074520 con 0x7f9c4c15db40 2024-02-18T11:47:45.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.677+0000 7f9c42ffd640 1 -- 172.21.15.116:0/1952280271 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9c440595e0 con 0x7f9c4c15db40 2024-02-18T11:47:45.671 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c3006bc70 msgr2=0x7f9c3006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c3006bc70 0x7f9c3006e130 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c036000 tx=0x7f9c3c03c040 comp rx=0 tx=0).stop 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 msgr2=0x7f9c4c15f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c15f5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400efc0 tx=0x7f9c4400c490 comp rx=0 tx=0).stop 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 shutdown_connections 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c3006bc70 0x7f9c3006e130 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c4c15db40 0x7f9c4c15f5a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c4c0b5880 0x7f9c4c15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 --2- 172.21.15.116:0/1952280271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c4c0b4bc0 0x7f9c4c14bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 >> 172.21.15.116:0/1952280271 conn(0x7f9c4c06f460 msgr2=0x7f9c4c0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 shutdown_connections 2024-02-18T11:47:45.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.679+0000 7f9c54593640 1 -- 172.21.15.116:0/1952280271 wait complete. 2024-02-18T11:47:45.683 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:47:45.683 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:47:45.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:45.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.846+0000 7fbed9688640 1 Processor -- start 2024-02-18T11:47:45.839 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.846+0000 7fbed9688640 1 -- start start 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed9688640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbed40d1080 0x7fbed40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed9688640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed40d3730 con 0x7fbed415dc60 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed9688640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed40cf250 con 0x7fbed4161850 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed9688640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed40ce9b0 con 0x7fbed40d1080 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbeca7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbed40d1080 0x7fbed40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed2ffd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbed415dc60 0x7fbed40d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50142/0 (socket says 172.21.15.116:50142) 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed2ffd640 1 -- 172.21.15.116:0/3001989809 learned_addr learned my addr 172.21.15.116:0/3001989809 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:45.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed27fc640 1 -- 172.21.15.116:0/3001989809 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463813200 0 0) 0x7fbed40d3730 con 0x7fbed415dc60 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbed27fc640 1 -- 172.21.15.116:0/3001989809 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec0003520 con 0x7fbed415dc60 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbeca7fc640 1 -- 172.21.15.116:0/3001989809 >> v1:172.21.15.143:6789/0 conn(0x7fbed4161850 legacy=0x7fbed40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbeca7fc640 1 -- 172.21.15.116:0/3001989809 >> v1:172.21.15.116:6789/0 conn(0x7fbed415dc60 legacy=0x7fbed40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.847+0000 7fbeca7fc640 1 -- 172.21.15.116:0/3001989809 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed40cec90 con 0x7fbed40d1080 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbeca7fc640 1 --2- 172.21.15.116:0/3001989809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed40b6c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbed40d38c0 tx=0x7fbebc02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e77eca74cc4d1e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed27fc640 1 -- 172.21.15.116:0/3001989809 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc030800 con 0x7fbed40d1080 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed27fc640 1 -- 172.21.15.116:0/3001989809 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc030960 con 0x7fbed40d1080 2024-02-18T11:47:45.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed27fc640 1 -- 172.21.15.116:0/3001989809 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc030cf0 con 0x7fbed40d1080 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 -- 172.21.15.116:0/3001989809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 msgr2=0x7fbed40b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 --2- 172.21.15.116:0/3001989809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed40b6c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbed40d38c0 tx=0x7fbebc02b940 comp rx=0 tx=0).stop 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 -- 172.21.15.116:0/3001989809 shutdown_connections 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 --2- 172.21.15.116:0/3001989809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed40b6c10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 -- 172.21.15.116:0/3001989809 >> 172.21.15.116:0/3001989809 conn(0x7fbed40b5910 msgr2=0x7fbed40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 -- 172.21.15.116:0/3001989809 shutdown_connections 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.848+0000 7fbed9688640 1 -- 172.21.15.116:0/3001989809 wait complete. 2024-02-18T11:47:45.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 Processor -- start 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 -- start start 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed41f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbed415dc60 0x7fbed41f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbed4161850 0x7fbed41f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4160af0 con 0x7fbed40d1080 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed415fee0 con 0x7fbed4161850 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.850+0000 7fbed9688640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed41601c0 con 0x7fbed415dc60 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed41f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed41f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34696/0 (socket says 172.21.15.116:34696) 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 -- 172.21.15.116:0/3363848463 learned_addr learned my addr 172.21.15.116:0/3363848463 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 -- 172.21.15.116:0/3363848463 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbed415dc60 msgr2=0x7fbed41f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbed415dc60 0x7fbed41f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 -- 172.21.15.116:0/3363848463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbed4161850 msgr2=0x7fbed41f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:45.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbed4161850 0x7fbed41f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 -- 172.21.15.116:0/3363848463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc009400 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed2ffd640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed41f32e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c500 tx=0x7fbec400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4005ce0 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4005e40 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4007330 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed415e1c0 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.851+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed4160780 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.852+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed4067840 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.853+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fbec4007550 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.853+0000 7fbecbfff640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbeb406bc70 0x7fbeb406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.853+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbec408a100 con 0x7fbed40d1080 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.853+0000 7fbeca7fc640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbeb406bc70 0x7fbeb406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:45.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.854+0000 7fbeca7fc640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbeb406bc70 0x7fbeb406e130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbebc036000 tx=0x7fbebc03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:45.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:45.857+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbec40546f0 con 0x7fbed40d1080 2024-02-18T11:47:46.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.027+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbed4074520 con 0x7fbed40d1080 2024-02-18T11:47:46.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.028+0000 7fbecbfff640 1 -- 172.21.15.116:0/3363848463 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fbec4057db0 con 0x7fbed40d1080 2024-02-18T11:47:46.021 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbeb406bc70 msgr2=0x7fbeb406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbeb406bc70 0x7fbeb406e130 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbebc036000 tx=0x7fbebc03b040 comp rx=0 tx=0).stop 2024-02-18T11:47:46.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 msgr2=0x7fbed41f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed41f32e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbec400c500 tx=0x7fbec400c9c0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 shutdown_connections 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbeb406bc70 0x7fbeb406e130 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbed4161850 0x7fbed41f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbed415dc60 0x7fbed41f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 --2- 172.21.15.116:0/3363848463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40d1080 0x7fbed41f32e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 >> 172.21.15.116:0/3363848463 conn(0x7fbed40b5910 msgr2=0x7fbed4153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 shutdown_connections 2024-02-18T11:47:46.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.030+0000 7fbed9688640 1 -- 172.21.15.116:0/3363848463 wait complete. 2024-02-18T11:47:46.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:47:46.035 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:47:46.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.144+0000 7f1588481640 1 Processor -- start 2024-02-18T11:47:46.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.144+0000 7f1588481640 1 -- start start 2024-02-18T11:47:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1588481640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f15800b31a0 0x7f158006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1588481640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15800b3610 con 0x7f158006e870 2024-02-18T11:47:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1588481640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f158006eca0 con 0x7f1580144090 2024-02-18T11:47:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1588481640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15800b2a50 con 0x7f15800b31a0 2024-02-18T11:47:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1586a1e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f158006e870 0x7f158015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50154/0 (socket says 172.21.15.116:50154) 2024-02-18T11:47:46.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1586a1e640 1 -- 172.21.15.116:0/3821303321 learned_addr learned my addr 172.21.15.116:0/3821303321 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f1585a1c640 1 --2- 172.21.15.116:0/3821303321 >> v2:172.21.15.143:3300/0 conn(0x7f15800b31a0 0x7f158006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861721036 0 0) 0x7f15800b3610 con 0x7f158006e870 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.145+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c003520 con 0x7f158006e870 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1064748692 0 0) 0x7f156c003520 con 0x7f158006e870 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 >> v2:172.21.15.143:3300/0 conn(0x7f15800b31a0 msgr2=0x7f158006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 --2- 172.21.15.116:0/3821303321 >> v2:172.21.15.143:3300/0 conn(0x7f15800b31a0 0x7f158006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 >> v1:172.21.15.143:6789/0 conn(0x7f1580144090 legacy=0x7f15800cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800cd0f0 con 0x7f158006e870 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f1578001670 con 0x7f158006e870 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 >> v1:172.21.15.116:6789/0 conn(0x7f158006e870 legacy=0x7f158015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f156c003970 0x7f156c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f156c0047c0 0x7f156c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c005180 0x7f156c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15800b2a50 con 0x7f156c0047c0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00c120 con 0x7f156c005180 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c00c2e0 con 0x7f156c003970 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1585a1c640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f156c0047c0 0x7f156c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1586a1e640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c005180 0x7f156c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f158621d640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f156c003970 0x7f156c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1585a1c640 1 -- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f156c003970 msgr2=0x7f156c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1585a1c640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f156c003970 0x7f156c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1585a1c640 1 -- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c005180 msgr2=0x7f156c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1585a1c640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c005180 0x7f156c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1586a1e640 1 -- 172.21.15.116:0/3821303321 reap_dead start 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.146+0000 7f1585a1c640 1 -- 172.21.15.116:0/3821303321 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c003520 con 0x7f156c0047c0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.147+0000 7f1585a1c640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f156c0047c0 0x7f156c004c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1570009650 tx=0x7f156c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a971281fe2649b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.147+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157002e8c0 con 0x7f156c0047c0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.147+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157002ea20 con 0x7f156c0047c0 2024-02-18T11:47:46.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.147+0000 7f158521b640 1 -- 172.21.15.116:0/3821303321 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157002edb0 con 0x7f156c0047c0 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 -- 172.21.15.116:0/3821303321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f156c0047c0 msgr2=0x7f156c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f156c0047c0 0x7f156c004c40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1570009650 tx=0x7f156c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 -- 172.21.15.116:0/3821303321 shutdown_connections 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 --2- 172.21.15.116:0/3821303321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f156c0047c0 0x7f156c004c40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 -- 172.21.15.116:0/3821303321 >> 172.21.15.116:0/3821303321 conn(0x7f1580067000 msgr2=0x7f158006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 -- 172.21.15.116:0/3821303321 shutdown_connections 2024-02-18T11:47:46.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.148+0000 7f1588481640 1 -- 172.21.15.116:0/3821303321 wait complete. 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.149+0000 7f1588481640 1 Processor -- start 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.149+0000 7f1588481640 1 -- start start 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c0047c0 0x7f1580141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f15800cada0 0x7f158015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 0x7f15800a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158015d860 con 0x7f158015d3e0 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15800a94a0 con 0x7f156c0047c0 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15800a91c0 con 0x7f15800cada0 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 0x7f15800a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 0x7f15800a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34718/0 (socket says 172.21.15.116:34718) 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 -- 172.21.15.116:0/3249362368 learned_addr learned my addr 172.21.15.116:0/3249362368 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1585a1c640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f15800cada0 0x7f158015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f158621d640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c0047c0 0x7f1580141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 -- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f15800cada0 msgr2=0x7f158015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f15800cada0 0x7f158015cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 -- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c0047c0 msgr2=0x7f1580141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c0047c0 0x7f1580141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 -- 172.21.15.116:0/3249362368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1580140a30 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1586a1e640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 0x7f15800a84b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15800cc360 tx=0x7f1578007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578001b40 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578001600 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157800fa70 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1580140d10 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.150+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15800cb200 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.151+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15800cbe40 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.152+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f157800fbd0 con 0x7f158015d3e0 2024-02-18T11:47:46.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.152+0000 7f1576ffd640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f156406bca0 0x7f156406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.152+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f157808a120 con 0x7f158015d3e0 2024-02-18T11:47:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.152+0000 7f158621d640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f156406bca0 0x7f156406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.153+0000 7f158621d640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f156406bca0 0x7f156406e160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f157c000f80 tx=0x7f157c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.156+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15780545c0 con 0x7f158015d3e0 2024-02-18T11:47:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.327+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1580141230 con 0x7f158015d3e0 2024-02-18T11:47:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.327+0000 7f1576ffd640 1 -- 172.21.15.116:0/3249362368 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1580141230 con 0x7f158015d3e0 2024-02-18T11:47:46.321 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f156406bca0 msgr2=0x7f156406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f156406bca0 0x7f156406e160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f157c000f80 tx=0x7f157c009040 comp rx=0 tx=0).stop 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 msgr2=0x7f15800a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 0x7f15800a84b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f15800cc360 tx=0x7f1578007720 comp rx=0 tx=0).stop 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 shutdown_connections 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f156406bca0 0x7f156406e160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f158015d3e0 0x7f15800a84b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f15800cada0 0x7f158015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 --2- 172.21.15.116:0/3249362368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156c0047c0 0x7f1580141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 >> 172.21.15.116:0/3249362368 conn(0x7f1580067000 msgr2=0x7f15800711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 shutdown_connections 2024-02-18T11:47:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.329+0000 7f1588481640 1 -- 172.21.15.116:0/3249362368 wait complete. 2024-02-18T11:47:46.333 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:47:46.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.445+0000 7f3b40ee1640 1 Processor -- start 2024-02-18T11:47:46.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.445+0000 7f3b40ee1640 1 -- start start 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b40ee1640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 0x7f3b3c071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b40ee1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3b3c0cf0e0 0x7f3b3c06b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b40ee1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c072370 con 0x7f3b3c0726c0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b40ee1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c0cdc80 con 0x7f3b3c0cf0e0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b40ee1640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c15c500 con 0x7f3b3c071a50 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b31d9b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3b3c0726c0 0x7f3b3c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50156/0 (socket says 172.21.15.116:50156) 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b31d9b640 1 -- 172.21.15.116:0/211142349 learned_addr learned my addr 172.21.15.116:0/211142349 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572888204 0 0) 0x7f3b3c072370 con 0x7f3b3c0726c0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b28003520 con 0x7f3b3c0726c0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 0x7f3b3c071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b3ad9d640 1 --2- 172.21.15.116:0/211142349 >> v2:172.21.15.143:3300/0 conn(0x7f3b3c0cf0e0 0x7f3b3c06b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1718984251 0 0) 0x7f3b28003520 con 0x7f3b3c0726c0 2024-02-18T11:47:46.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 msgr2=0x7f3b3c071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 0x7f3b3c071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 >> v2:172.21.15.143:3300/0 conn(0x7f3b3c0cf0e0 msgr2=0x7f3b3c06b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 --2- 172.21.15.116:0/211142349 >> v2:172.21.15.143:3300/0 conn(0x7f3b3c0cf0e0 0x7f3b3c06b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c15c7e0 con 0x7f3b3c0726c0 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f3b24001670 con 0x7f3b3c0726c0 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 >> v1:172.21.15.116:6789/0 conn(0x7f3b3c0726c0 legacy=0x7f3b3c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b28003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b280047c0 0x7f3b28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 0x7f3b28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0cdc80 con 0x7f3b28003970 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c15c500 con 0x7f3b280047c0 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2800b980 con 0x7f3b3c071a50 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.446+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b28003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b31d9b640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b280047c0 0x7f3b28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3ad9d640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 0x7f3b28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3a59c640 1 -- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 msgr2=0x7f3b28005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c071a50 0x7f3b28005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3a59c640 1 -- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b280047c0 msgr2=0x7f3b28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b280047c0 0x7f3b28004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3a59c640 1 -- 172.21.15.116:0/211142349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28003520 con 0x7f3b28003970 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.447+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b28003d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009ac0 tx=0x7f3b280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12671f4c170c742 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c02c6b0 con 0x7f3b28003970 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c02c810 con 0x7f3b28003970 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b39d9b640 1 -- 172.21.15.116:0/211142349 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c02cba0 con 0x7f3b28003970 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b40ee1640 1 -- 172.21.15.116:0/211142349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 msgr2=0x7f3b28003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/211142349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b28003d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009ac0 tx=0x7f3b280057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b3ad9d640 1 -- 172.21.15.116:0/211142349 reap_dead start 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b40ee1640 1 -- 172.21.15.116:0/211142349 shutdown_connections 2024-02-18T11:47:46.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b40ee1640 1 -- 172.21.15.116:0/211142349 >> 172.21.15.116:0/211142349 conn(0x7f3b3c0cfd80 msgr2=0x7f3b3c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b40ee1640 1 -- 172.21.15.116:0/211142349 shutdown_connections 2024-02-18T11:47:46.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.448+0000 7f3b40ee1640 1 -- 172.21.15.116:0/211142349 wait complete. 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 Processor -- start 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 -- start start 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b3c152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b3c15a0d0 0x7f3b3c0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c15a700 0x7f3b3c15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c15b0d0 con 0x7f3b28003970 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0ad700 con 0x7f3b3c15a0d0 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b40ee1640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0ad9e0 con 0x7f3b3c15a700 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b3a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b3c152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b3a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b3c152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34740/0 (socket says 172.21.15.116:34740) 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.450+0000 7f3b3a59c640 1 -- 172.21.15.116:0/1395378433 learned_addr learned my addr 172.21.15.116:0/1395378433 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3ad9d640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c15a700 0x7f3b3c15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b31d9b640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b3c15a0d0 0x7f3b3c0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3a59c640 1 -- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c15a700 msgr2=0x7f3b3c15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c15a700 0x7f3b3c15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3a59c640 1 -- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b3c15a0d0 msgr2=0x7f3b3c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b3c15a0d0 0x7f3b3c0acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3a59c640 1 -- 172.21.15.116:0/1395378433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0adcc0 con 0x7f3b28003970 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b31d9b640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b3c15a0d0 0x7f3b3c0acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b3a59c640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b3c152ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009ac0 tx=0x7f3b2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c003810 con 0x7f3b28003970 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c003970 con 0x7f3b28003970 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c0adfa0 con 0x7f3b28003970 2024-02-18T11:47:46.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c003d20 con 0x7f3b28003970 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.451+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c0ad2d0 con 0x7f3b28003970 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.452+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c067180 con 0x7f3b28003970 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.452+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3b2c040450 con 0x7f3b28003970 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.453+0000 7f3b337fe640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3b1c06bca0 0x7f3b1c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.453+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3b2c0acaa0 con 0x7f3b28003970 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.453+0000 7f3b31d9b640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3b1c06bca0 0x7f3b1c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.453+0000 7f3b31d9b640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3b1c06bca0 0x7f3b1c06e160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c1533e0 tx=0x7f3b24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.457+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b2c076f50 con 0x7f3b28003970 2024-02-18T11:47:46.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.635+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b3c0ad460 con 0x7f3b28003970 2024-02-18T11:47:46.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.635+0000 7f3b337fe640 1 -- 172.21.15.116:0/1395378433 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b2c07a610 con 0x7f3b28003970 2024-02-18T11:47:46.629 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:47:46.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3b1c06bca0 msgr2=0x7f3b1c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3b1c06bca0 0x7f3b1c06e160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c1533e0 tx=0x7f3b24009040 comp rx=0 tx=0).stop 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 msgr2=0x7f3b3c152ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b3c152ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009ac0 tx=0x7f3b2c004730 comp rx=0 tx=0).stop 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 shutdown_connections 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3b1c06bca0 0x7f3b1c06e160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3b3c15a700 0x7f3b3c15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b3c15a0d0 0x7f3b3c0acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 --2- 172.21.15.116:0/1395378433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b28003970 0x7f3b3c152ea0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 >> 172.21.15.116:0/1395378433 conn(0x7f3b3c0cfd80 msgr2=0x7f3b3c0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 shutdown_connections 2024-02-18T11:47:46.631 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.637+0000 7f3b40ee1640 1 -- 172.21.15.116:0/1395378433 wait complete. 2024-02-18T11:47:46.641 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:47:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.751+0000 7fd98761b640 1 Processor -- start 2024-02-18T11:47:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.751+0000 7fd98761b640 1 -- start start 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd98761b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd98015dbf0 0x7fd9801617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd98761b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800b9370 0x7fd9800ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd98761b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9800cec80 0x7fd98006c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd98761b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9800cf110 con 0x7fd9800cec80 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd98761b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9800d1ee0 con 0x7fd98015dbf0 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd98761b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd980161dc0 con 0x7fd9800b9370 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9800cec80 0x7fd98006c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9800cec80 0x7fd98006c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34750/0 (socket says 172.21.15.116:34750) 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 -- 172.21.15.116:0/1994799851 learned_addr learned my addr 172.21.15.116:0/1994799851 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 -- 172.21.15.116:0/1994799851 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800b9370 msgr2=0x7fd9800ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 --2- 172.21.15.116:0/1994799851 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800b9370 0x7fd9800ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 -- 172.21.15.116:0/1994799851 >> v2:172.21.15.143:3300/0 conn(0x7fd98015dbf0 msgr2=0x7fd9801617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 --2- 172.21.15.116:0/1994799851 >> v2:172.21.15.143:3300/0 conn(0x7fd98015dbf0 0x7fd9801617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.752+0000 7fd985bb8640 1 -- 172.21.15.116:0/1994799851 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9801620a0 con 0x7fd9800cec80 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.753+0000 7fd985bb8640 1 --2- 172.21.15.116:0/1994799851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9800cec80 0x7fd98006c210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd98006d330 tx=0x7fd97802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f77d6bcab878e09 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.753+0000 7fd977fff640 1 -- 172.21.15.116:0/1994799851 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97802ca70 con 0x7fd9800cec80 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.753+0000 7fd977fff640 1 -- 172.21.15.116:0/1994799851 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97802cbd0 con 0x7fd9800cec80 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.753+0000 7fd977fff640 1 -- 172.21.15.116:0/1994799851 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780343f0 con 0x7fd9800cec80 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 -- 172.21.15.116:0/1994799851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9800cec80 msgr2=0x7fd98006c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 --2- 172.21.15.116:0/1994799851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9800cec80 0x7fd98006c210 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd98006d330 tx=0x7fd97802bed0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 -- 172.21.15.116:0/1994799851 shutdown_connections 2024-02-18T11:47:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 --2- 172.21.15.116:0/1994799851 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800b9370 0x7fd9800ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 --2- 172.21.15.116:0/1994799851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9800cec80 0x7fd98006c210 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 --2- 172.21.15.116:0/1994799851 >> v2:172.21.15.143:3300/0 conn(0x7fd98015dbf0 0x7fd9801617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 -- 172.21.15.116:0/1994799851 >> 172.21.15.116:0/1994799851 conn(0x7fd9800b5480 msgr2=0x7fd9800b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 -- 172.21.15.116:0/1994799851 shutdown_connections 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.754+0000 7fd98761b640 1 -- 172.21.15.116:0/1994799851 wait complete. 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.755+0000 7fd98761b640 1 Processor -- start 2024-02-18T11:47:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.755+0000 7fd98761b640 1 -- start start 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9800b9370 0x7fd98014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800cec80 0x7fd9801f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 0x7fd980159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98014b120 con 0x7fd98015dbf0 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980148470 con 0x7fd9800b9370 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd980148750 con 0x7fd9800cec80 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 0x7fd980159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 0x7fd980159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34760/0 (socket says 172.21.15.116:34760) 2024-02-18T11:47:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 -- 172.21.15.116:0/3475162477 learned_addr learned my addr 172.21.15.116:0/3475162477 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd9853b7640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9800b9370 0x7fd98014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd984bb6640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800cec80 0x7fd9801f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 -- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800cec80 msgr2=0x7fd9801f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800cec80 0x7fd9801f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 -- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9800b9370 msgr2=0x7fd98014abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9800b9370 0x7fd98014abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 -- 172.21.15.116:0/3475162477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd978009b30 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd985bb8640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 0x7fd980159960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd978036040 tx=0x7fd978004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978009510 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97803e450 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98015e150 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.756+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97803e7e0 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.757+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd980148d10 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.758+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980067840 con 0x7fd98015dbf0 2024-02-18T11:47:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.758+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd97803e940 con 0x7fd98015dbf0 2024-02-18T11:47:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.758+0000 7fd975ffb640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd96406bc70 0x7fd96406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.759+0000 7fd9853b7640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd96406bc70 0x7fd96406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.759+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd9780ad840 con 0x7fd98015dbf0 2024-02-18T11:47:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.759+0000 7fd9853b7640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd96406bc70 0x7fd96406e130 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098e0 tx=0x7fd97c009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:46.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.763+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd978077e00 con 0x7fd98015dbf0 2024-02-18T11:47:46.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.937+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd980074520 con 0x7fd98015dbf0 2024-02-18T11:47:46.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.937+0000 7fd975ffb640 1 -- 172.21.15.116:0/3475162477 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd97807b4c0 con 0x7fd98015dbf0 2024-02-18T11:47:46.931 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:47:46.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd96406bc70 msgr2=0x7fd96406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd96406bc70 0x7fd96406e130 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0098e0 tx=0x7fd97c009060 comp rx=0 tx=0).stop 2024-02-18T11:47:46.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 msgr2=0x7fd980159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:46.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 0x7fd980159960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd978036040 tx=0x7fd978004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:46.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 shutdown_connections 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd96406bc70 0x7fd96406e130 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd98015dbf0 0x7fd980159960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9800cec80 0x7fd9801f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.939+0000 7fd98761b640 1 --2- 172.21.15.116:0/3475162477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9800b9370 0x7fd98014abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.940+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 >> 172.21.15.116:0/3475162477 conn(0x7fd9800b5480 msgr2=0x7fd9800d2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.940+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 shutdown_connections 2024-02-18T11:47:46.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:46.940+0000 7fd98761b640 1 -- 172.21.15.116:0/3475162477 wait complete. 2024-02-18T11:47:46.944 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:47:47.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.054+0000 7fa24c60e640 1 Processor -- start 2024-02-18T11:47:47.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.054+0000 7fa24c60e640 1 -- start start 2024-02-18T11:47:47.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.055+0000 7fa24c60e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2440719d0 0x7fa2440b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.055+0000 7fa24c60e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2440ceed0 con 0x7fa2440712f0 2024-02-18T11:47:47.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.055+0000 7fa24c60e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2440b1100 con 0x7fa244145250 2024-02-18T11:47:47.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.055+0000 7fa24c60e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2440ce2c0 con 0x7fa2440719d0 2024-02-18T11:47:47.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.055+0000 7fa249ba9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa2440712f0 0x7fa2440b4680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50170/0 (socket says 172.21.15.116:50170) 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.055+0000 7fa249ba9640 1 -- 172.21.15.116:0/2269447559 learned_addr learned my addr 172.21.15.116:0/2269447559 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa24abab640 1 --2- 172.21.15.116:0/2269447559 >> v2:172.21.15.143:3300/0 conn(0x7fa2440719d0 0x7fa2440b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138941510 0 0) 0x7fa2440ceed0 con 0x7fa2440712f0 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa230003520 con 0x7fa2440712f0 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744323516 0 0) 0x7fa230003520 con 0x7fa2440712f0 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 >> v2:172.21.15.143:3300/0 conn(0x7fa2440719d0 msgr2=0x7fa2440b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 --2- 172.21.15.116:0/2269447559 >> v2:172.21.15.143:3300/0 conn(0x7fa2440719d0 0x7fa2440b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 >> v1:172.21.15.143:6789/0 conn(0x7fa244145250 legacy=0x7fa244070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2440ce5a0 con 0x7fa2440712f0 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fa234001670 con 0x7fa2440712f0 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa24abab640 1 --2- 172.21.15.116:0/2269447559 >> v2:172.21.15.143:3300/0 conn(0x7fa2440719d0 0x7fa2440b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 >> v1:172.21.15.116:6789/0 conn(0x7fa2440712f0 legacy=0x7fa2440b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa230003970 0x7fa230003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2300047c0 0x7fa230004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa230005180 0x7fa23000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2440ce2c0 con 0x7fa2300047c0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23000c120 con 0x7fa230005180 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23000c2e0 con 0x7fa230003970 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa249ba9640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2300047c0 0x7fa230004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa24abab640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa230005180 0x7fa23000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa24a3aa640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa230003970 0x7fa230003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa249ba9640 1 -- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa230003970 msgr2=0x7fa230003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa249ba9640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa230003970 0x7fa230003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa249ba9640 1 -- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa230005180 msgr2=0x7fa23000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa249ba9640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa230005180 0x7fa23000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa24abab640 1 -- 172.21.15.116:0/2269447559 reap_dead start 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa249ba9640 1 -- 172.21.15.116:0/2269447559 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa230003520 con 0x7fa2300047c0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.056+0000 7fa24a3aa640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa230003970 0x7fa230003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.057+0000 7fa249ba9640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2300047c0 0x7fa230004c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa23400c240 tx=0x7fa2300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c973abcaec5be8cf server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.057+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234005ad0 con 0x7fa2300047c0 2024-02-18T11:47:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.057+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa234005c30 con 0x7fa2300047c0 2024-02-18T11:47:47.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.057+0000 7fa2493a8640 1 -- 172.21.15.116:0/2269447559 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23402ca90 con 0x7fa2300047c0 2024-02-18T11:47:47.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 -- 172.21.15.116:0/2269447559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2300047c0 msgr2=0x7fa230004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2300047c0 0x7fa230004c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa23400c240 tx=0x7fa2300057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 -- 172.21.15.116:0/2269447559 shutdown_connections 2024-02-18T11:47:47.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 --2- 172.21.15.116:0/2269447559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2300047c0 0x7fa230004c40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 -- 172.21.15.116:0/2269447559 >> 172.21.15.116:0/2269447559 conn(0x7fa24406f460 msgr2=0x7fa24406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:47.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 -- 172.21.15.116:0/2269447559 shutdown_connections 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.058+0000 7fa24c60e640 1 -- 172.21.15.116:0/2269447559 wait complete. 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.059+0000 7fa24c60e640 1 Processor -- start 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.059+0000 7fa24c60e640 1 -- start start 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2300047c0 0x7fa2441e6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2441d1df0 0x7fa24413ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 0x7fa24413b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441d0d50 con 0x7fa2441d2270 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441d1030 con 0x7fa2300047c0 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2441d1310 con 0x7fa2441d1df0 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 0x7fa24413b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 0x7fa24413b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34770/0 (socket says 172.21.15.116:34770) 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 -- 172.21.15.116:0/3723069690 learned_addr learned my addr 172.21.15.116:0/3723069690 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa249ba9640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2441d1df0 0x7fa24413ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24a3aa640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2300047c0 0x7fa2441e6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 -- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2441d1df0 msgr2=0x7fa24413ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2441d1df0 0x7fa24413ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 -- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2300047c0 msgr2=0x7fa2441e6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2300047c0 0x7fa2441e6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 -- 172.21.15.116:0/3723069690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2441d15f0 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24abab640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 0x7fa24413b170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa244074430 tx=0x7fa24000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24001c070 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000f040 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000e040 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2441d18d0 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.060+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2441e7150 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.061+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2440cf7a0 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.062+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa240003990 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.062+0000 7fa23affd640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa22806bc70 0x7fa22806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.062+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa24008aba0 con 0x7fa2441d2270 2024-02-18T11:47:47.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.062+0000 7fa24a3aa640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa22806bc70 0x7fa22806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.063+0000 7fa24a3aa640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa22806bc70 0x7fa22806e130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:47.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.066+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa240055180 con 0x7fa2441d2270 2024-02-18T11:47:47.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.238+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa2441e7150 con 0x7fa2441d2270 2024-02-18T11:47:47.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.239+0000 7fa23affd640 1 -- 172.21.15.116:0/3723069690 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2441e7150 con 0x7fa2441d2270 2024-02-18T11:47:47.232 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.240+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa22806bc70 msgr2=0x7fa22806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.240+0000 7fa24c60e640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa22806bc70 0x7fa22806e130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c009040 comp rx=0 tx=0).stop 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.240+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 msgr2=0x7fa24413b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.240+0000 7fa24c60e640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 0x7fa24413b170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa244074430 tx=0x7fa24000f580 comp rx=0 tx=0).stop 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 shutdown_connections 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa22806bc70 0x7fa22806e130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2441d2270 0x7fa24413b170 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2441d1df0 0x7fa24413ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 --2- 172.21.15.116:0/3723069690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2300047c0 0x7fa2441e6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 >> 172.21.15.116:0/3723069690 conn(0x7fa24406f460 msgr2=0x7fa2440b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:47.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 shutdown_connections 2024-02-18T11:47:47.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.241+0000 7fa24c60e640 1 -- 172.21.15.116:0/3723069690 wait complete. 2024-02-18T11:47:47.245 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:47:47.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 Processor -- start 2024-02-18T11:47:47.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 -- start start 2024-02-18T11:47:47.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f405015da10 0x7f40500d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f40500d33d0 0x7f40500d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500b79f0 con 0x7f40500b95f0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40500d2330 con 0x7f405015da10 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404f59e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500d2610 con 0x7f40500d33d0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.355+0000 7f404ed9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f40500b95f0 0x7f40500d3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:50182/0 (socket says 172.21.15.116:50182) 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 -- 172.21.15.116:0/886483291 learned_addr learned my addr 172.21.15.116:0/886483291 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714789538 0 0) 0x7f40500b79f0 con 0x7f40500b95f0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f403c003520 con 0x7f40500b95f0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f4047fff640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f40500d33d0 0x7f40500d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404e59c640 1 --2- 172.21.15.116:0/886483291 >> v2:172.21.15.143:3300/0 conn(0x7f405015da10 0x7f40500d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4092131016 0 0) 0x7f403c003520 con 0x7f40500b95f0 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f40500d33d0 msgr2=0x7f40500d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f40500d33d0 0x7f40500d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 >> v2:172.21.15.143:3300/0 conn(0x7f405015da10 msgr2=0x7f40500d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 --2- 172.21.15.116:0/886483291 >> v2:172.21.15.143:3300/0 conn(0x7f405015da10 0x7f40500d19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500d28f0 con 0x7f40500b95f0 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f40480019f0 con 0x7f40500b95f0 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404e59c640 1 --2- 172.21.15.116:0/886483291 >> v2:172.21.15.143:3300/0 conn(0x7f405015da10 0x7f40500d19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 >> v1:172.21.15.116:6789/0 conn(0x7f40500b95f0 legacy=0x7f40500d3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c003970 0x7f403c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f403c0047c0 0x7f403c004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f403c005140 0x7f403c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500d2330 con 0x7f403c005140 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500d2610 con 0x7f403c003970 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c00bec0 con 0x7f403c0047c0 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f403c005140 0x7f403c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f4047fff640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f403c0047c0 0x7f403c004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404e59c640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c003970 0x7f403c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 -- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f403c0047c0 msgr2=0x7f403c004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f403c0047c0 0x7f403c004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 -- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c003970 msgr2=0x7f403c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c003970 0x7f403c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.356+0000 7f404ed9d640 1 -- 172.21.15.116:0/886483291 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c003520 con 0x7f403c005140 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.357+0000 7f404ed9d640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f403c005140 0x7f403c00b980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f404800a830 tx=0x7f403c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f60b6b80f77a9dc server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.357+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048034630 con 0x7f403c005140 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.357+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404802f070 con 0x7f403c005140 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.357+0000 7f404dd9b640 1 -- 172.21.15.116:0/886483291 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048034a10 con 0x7f403c005140 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404f59e640 1 -- 172.21.15.116:0/886483291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f403c005140 msgr2=0x7f403c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404f59e640 1 --2- 172.21.15.116:0/886483291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f403c005140 0x7f403c00b980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f404800a830 tx=0x7f403c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404ed9d640 1 -- 172.21.15.116:0/886483291 reap_dead start 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404f59e640 1 -- 172.21.15.116:0/886483291 shutdown_connections 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404f59e640 1 -- 172.21.15.116:0/886483291 >> 172.21.15.116:0/886483291 conn(0x7f405006c440 msgr2=0x7f405006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404f59e640 1 -- 172.21.15.116:0/886483291 shutdown_connections 2024-02-18T11:47:47.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.358+0000 7f404f59e640 1 -- 172.21.15.116:0/886483291 wait complete. 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.359+0000 7f404f59e640 1 Processor -- start 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.359+0000 7f404f59e640 1 -- start start 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404f59e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c005140 0x7f405014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404f59e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4050150080 0x7f40501f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404f59e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 0x7f40501f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404f59e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050155ba0 con 0x7f40501e66c0 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404f59e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500cb130 con 0x7f403c005140 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404f59e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500cb3d0 con 0x7f4050150080 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 0x7f40501f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 0x7f40501f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34792/0 (socket says 172.21.15.116:34792) 2024-02-18T11:47:47.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 -- 172.21.15.116:0/4211136156 learned_addr learned my addr 172.21.15.116:0/4211136156 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f4047fff640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4050150080 0x7f40501f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404e59c640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c005140 0x7f405014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 -- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4050150080 msgr2=0x7f40501f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4050150080 0x7f40501f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 -- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c005140 msgr2=0x7f405014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c005140 0x7f405014dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 -- 172.21.15.116:0/4211136156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500cb6b0 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.360+0000 7f404ed9d640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 0x7f40501f6bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4048001cb0 tx=0x7f40500b6930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.361+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048034630 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.361+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4048034790 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.361+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048034b00 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.361+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500cb990 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.361+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40500cbeb0 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.362+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405015c6b0 con 0x7f40501e66c0 2024-02-18T11:47:47.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.362+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4048031030 con 0x7f40501e66c0 2024-02-18T11:47:47.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.362+0000 7f4046ffd640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f403006bca0 0x7f403006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:47.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.362+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f40480ac7d0 con 0x7f40501e66c0 2024-02-18T11:47:47.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.363+0000 7f404e59c640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f403006bca0 0x7f403006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:47.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.363+0000 7f404e59c640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f403006bca0 0x7f403006e160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:47.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.367+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f404807a7a0 con 0x7f40501e66c0 2024-02-18T11:47:47.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.548+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f405015c6b0 con 0x7f40501e66c0 2024-02-18T11:47:47.542 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.549+0000 7f4046ffd640 1 -- 172.21.15.116:0/4211136156 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4048001fc0 con 0x7f40501e66c0 2024-02-18T11:47:47.542 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T11:47:47.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f403006bca0 msgr2=0x7f403006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f403006bca0 0x7f403006e160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040008040 comp rx=0 tx=0).stop 2024-02-18T11:47:47.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 msgr2=0x7f40501f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:47.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 0x7f40501f6bb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4048001cb0 tx=0x7f40500b6930 comp rx=0 tx=0).stop 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 shutdown_connections 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f403006bca0 0x7f403006e160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40501e66c0 0x7f40501f6bb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4050150080 0x7f40501f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 --2- 172.21.15.116:0/4211136156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403c005140 0x7f405014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 >> 172.21.15.116:0/4211136156 conn(0x7f405006c440 msgr2=0x7f40501597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 shutdown_connections 2024-02-18T11:47:47.545 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:47.551+0000 7f404f59e640 1 -- 172.21.15.116:0/4211136156 wait complete. 2024-02-18T11:47:47.555 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-02-18T11:47:47.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:47.555 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-18T11:47:47.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi116.stderr:ceph-fuse[45005]: fuse finished with error 0 and tester_r 0 2024-02-18T11:47:47.584 INFO:teuthology.orchestra.run:waiting for 300 2024-02-18T11:47:53.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:47:53.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:47:53.687 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-18T11:47:53.713 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:53.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.871+0000 7f26102f4640 1 Processor -- start 2024-02-18T11:47:53.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.871+0000 7f26102f4640 1 -- start start 2024-02-18T11:47:53.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.872+0000 7f26102f4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 0x7f26080d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:53.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.872+0000 7f26102f4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26080cf900 0x7f260806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:53.866 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.872+0000 7f26102f4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26080d0a80 con 0x7f260806cd80 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.872+0000 7f26102f4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f260815f390 con 0x7f26080cf900 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.872+0000 7f26102f4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26080cfe70 con 0x7f260815efb0 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26080cf900 0x7f260806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260e090640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 0x7f26080d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f26080cf900 0x7f260806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:55238/0 (socket says 172.21.15.116:55238) 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260e090640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 0x7f26080d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:49336/0 (socket says 172.21.15.116:49336) 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 -- 172.21.15.116:0/1831368078 learned_addr learned my addr 172.21.15.116:0/1831368078 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 -- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 msgr2=0x7f26080d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 --2- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 0x7f26080d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 -- 172.21.15.116:0/1831368078 >> v1:172.21.15.143:6789/0 conn(0x7f260806cd80 legacy=0x7f2608099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d88f640 1 -- 172.21.15.116:0/1831368078 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26080d0150 con 0x7f26080cf900 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260e090640 1 --2- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 0x7f26080d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:53.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.873+0000 7f260d08e640 1 -- 172.21.15.116:0/1831368078 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912276342 0 0) 0x7f26080d0a80 con 0x7f260806cd80 2024-02-18T11:47:53.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.874+0000 7f260d88f640 1 --2- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26080cf900 0x7f260806c840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009870 tx=0x7f25f802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f43591cd811c1a89 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.874+0000 7f260d08e640 1 -- 172.21.15.116:0/1831368078 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8031640 con 0x7f26080cf900 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.874+0000 7f260d08e640 1 -- 172.21.15.116:0/1831368078 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f80317a0 con 0x7f26080cf900 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.874+0000 7f260d08e640 1 -- 172.21.15.116:0/1831368078 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f8031b30 con 0x7f26080cf900 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 -- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26080cf900 msgr2=0x7f260806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 --2- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26080cf900 0x7f260806c840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009870 tx=0x7f25f802b8f0 comp rx=0 tx=0).stop 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 -- 172.21.15.116:0/1831368078 shutdown_connections 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 --2- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26080cf900 0x7f260806c840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 --2- 172.21.15.116:0/1831368078 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260815efb0 0x7f26080d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 -- 172.21.15.116:0/1831368078 >> 172.21.15.116:0/1831368078 conn(0x7f26080d1790 msgr2=0x7f26080d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 -- 172.21.15.116:0/1831368078 shutdown_connections 2024-02-18T11:47:53.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.875+0000 7f26102f4640 1 -- 172.21.15.116:0/1831368078 wait complete. 2024-02-18T11:47:53.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.876+0000 7f26102f4640 1 Processor -- start 2024-02-18T11:47:53.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.876+0000 7f26102f4640 1 -- start start 2024-02-18T11:47:53.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f26102f4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260806cd80 0x7f26080b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f26102f4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 0x7f260815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f26102f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f260815efb0 0x7f260809c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f26102f4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26080b45e0 con 0x7f26080cf900 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f26102f4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26080b4e80 con 0x7f260815efb0 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f26102f4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260809cce0 con 0x7f260806cd80 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 0x7f260815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 0x7f260815ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40190/0 (socket says 172.21.15.116:40190) 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 -- 172.21.15.116:0/645982221 learned_addr learned my addr 172.21.15.116:0/645982221 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260e090640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260806cd80 0x7f26080b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260e891640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f260815efb0 0x7f260809c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 -- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260806cd80 msgr2=0x7f26080b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260806cd80 0x7f26080b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 -- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f260815efb0 msgr2=0x7f260809c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:53.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f260815efb0 0x7f260809c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 -- 172.21.15.116:0/645982221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604009a70 con 0x7f26080cf900 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.877+0000 7f260d88f640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 0x7f260815ddf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009870 tx=0x7f25f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.878+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f803a930 con 0x7f26080cf900 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.878+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f803aa90 con 0x7f26080cf900 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.878+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f80094d0 con 0x7f26080cf900 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.878+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f803ae40 con 0x7f26080cf900 2024-02-18T11:47:53.873 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.878+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26080b4b10 con 0x7f26080cf900 2024-02-18T11:47:53.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.879+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2608067840 con 0x7f26080cf900 2024-02-18T11:47:53.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.879+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f25f802f050 con 0x7f26080cf900 2024-02-18T11:47:53.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.879+0000 7f25feffd640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ec06bc70 0x7f25ec06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:53.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.879+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f25f80ac8b0 con 0x7f26080cf900 2024-02-18T11:47:53.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.879+0000 7f260e090640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ec06bc70 0x7f25ec06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:53.874 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.880+0000 7f260e090640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ec06bc70 0x7f25ec06e130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26040114e0 tx=0x7f260400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:53.877 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:53.884+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25f8076e90 con 0x7f26080cf900 2024-02-18T11:47:54.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.054+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26080b4ff0 con 0x7f26080cf900 2024-02-18T11:47:54.049 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.056+0000 7f25feffd640 1 -- 172.21.15.116:0/645982221 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f25f807a550 con 0x7f26080cf900 2024-02-18T11:47:54.049 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:54.049 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.057+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ec06bc70 msgr2=0x7f25ec06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.057+0000 7f26102f4640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ec06bc70 0x7f25ec06e130 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26040114e0 tx=0x7f260400b040 comp rx=0 tx=0).stop 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.057+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 msgr2=0x7f260815ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.057+0000 7f26102f4640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 0x7f260815ddf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009870 tx=0x7f25f80046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 shutdown_connections 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ec06bc70 0x7f25ec06e130 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f260815efb0 0x7f260809c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26080cf900 0x7f260815ddf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 --2- 172.21.15.116:0/645982221 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f260806cd80 0x7f26080b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 >> 172.21.15.116:0/645982221 conn(0x7f26080d1790 msgr2=0x7f26080ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:54.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 shutdown_connections 2024-02-18T11:47:54.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.058+0000 7f26102f4640 1 -- 172.21.15.116:0/645982221 wait complete. 2024-02-18T11:47:54.062 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:54.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.171+0000 7f740bf6c640 1 Processor -- start 2024-02-18T11:47:54.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.172+0000 7f740bf6c640 1 -- start start 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f740bf6c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74040b0750 0x7f74040b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f740bf6c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0b90 0x7f74040cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f740bf6c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74040b1070 con 0x7f74040d0730 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f740bf6c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74040ccb70 con 0x7f74040b0750 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f740bf6c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040cf970 con 0x7f74040d0b90 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f7409507640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f74040d0730 0x7f74040cbe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:44212/0 (socket says 172.21.15.116:44212) 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f7409507640 1 -- 172.21.15.116:0/287281765 learned_addr learned my addr 172.21.15.116:0/287281765 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f740a509640 1 --2- 172.21.15.116:0/287281765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0b90 0x7f74040cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f7409d08640 1 --2- 172.21.15.116:0/287281765 >> v2:172.21.15.143:3300/0 conn(0x7f74040b0750 0x7f74040b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f7408d06640 1 -- 172.21.15.116:0/287281765 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068823157 0 0) 0x7f74040b1070 con 0x7f74040d0730 2024-02-18T11:47:54.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.173+0000 7f7408d06640 1 -- 172.21.15.116:0/287281765 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f0003520 con 0x7f74040d0730 2024-02-18T11:47:54.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.174+0000 7f740a509640 1 -- 172.21.15.116:0/287281765 >> v2:172.21.15.143:3300/0 conn(0x7f74040b0750 msgr2=0x7f74040b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:54.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.174+0000 7f740a509640 1 --2- 172.21.15.116:0/287281765 >> v2:172.21.15.143:3300/0 conn(0x7f74040b0750 0x7f74040b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.174+0000 7f740a509640 1 -- 172.21.15.116:0/287281765 >> v1:172.21.15.116:6789/0 conn(0x7f74040d0730 legacy=0x7f74040cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.174+0000 7f7408d06640 1 -- 172.21.15.116:0/287281765 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1219854571 0 0) 0x7f73f0003520 con 0x7f74040d0730 2024-02-18T11:47:54.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.174+0000 7f740a509640 1 -- 172.21.15.116:0/287281765 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040cfc50 con 0x7f74040d0b90 2024-02-18T11:47:54.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.174+0000 7f740a509640 1 --2- 172.21.15.116:0/287281765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0b90 0x7f74040cc550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009d90 tx=0x7f73fc02bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7bfdf55ecc2ee3e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:54.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f7408d06640 1 -- 172.21.15.116:0/287281765 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4001670 con 0x7f74040d0b90 2024-02-18T11:47:54.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f7408d06640 1 -- 172.21.15.116:0/287281765 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc02c8a0 con 0x7f74040d0b90 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f7408d06640 1 -- 172.21.15.116:0/287281765 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc02cc30 con 0x7f74040d0b90 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 -- 172.21.15.116:0/287281765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0b90 msgr2=0x7f74040cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 --2- 172.21.15.116:0/287281765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0b90 0x7f74040cc550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009d90 tx=0x7f73fc02bd70 comp rx=0 tx=0).stop 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 -- 172.21.15.116:0/287281765 shutdown_connections 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 --2- 172.21.15.116:0/287281765 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0b90 0x7f74040cc550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 --2- 172.21.15.116:0/287281765 >> v2:172.21.15.143:3300/0 conn(0x7f74040b0750 0x7f74040b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 -- 172.21.15.116:0/287281765 >> 172.21.15.116:0/287281765 conn(0x7f74040b3380 msgr2=0x7f74040b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 -- 172.21.15.116:0/287281765 shutdown_connections 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.175+0000 7f740bf6c640 1 -- 172.21.15.116:0/287281765 wait complete. 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.176+0000 7f740bf6c640 1 Processor -- start 2024-02-18T11:47:54.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.176+0000 7f740bf6c640 1 -- start start 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740bf6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74040b0750 0x7f74041523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740bf6c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0730 0x7f7404154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740bf6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 0x7f74040cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740bf6c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74041f1250 con 0x7f7404152900 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740bf6c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74041f1530 con 0x7f74040b0750 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740bf6c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74041f1810 con 0x7f74040d0730 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 0x7f74040cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 0x7f74040cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40196/0 (socket says 172.21.15.116:40196) 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 -- 172.21.15.116:0/3968501458 learned_addr learned my addr 172.21.15.116:0/3968501458 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f7409507640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0730 0x7f7404154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f7409d08640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74040b0750 0x7f74041523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 -- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0730 msgr2=0x7f7404154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0730 0x7f7404154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 -- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74040b0750 msgr2=0x7f74041523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74040b0750 0x7f74041523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 -- 172.21.15.116:0/3968501458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc009a70 con 0x7f7404152900 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f7409507640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0730 0x7f7404154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f7409d08640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74040b0750 0x7f74041523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.178+0000 7f740a509640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 0x7f74040cade0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009d90 tx=0x7f73fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.179+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc034e50 con 0x7f7404152900 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.179+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc02ce90 con 0x7f7404152900 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.179+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74041f1af0 con 0x7f7404152900 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.179+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc040690 con 0x7f7404152900 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.179+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74041f1ff0 con 0x7f7404152900 2024-02-18T11:47:54.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.180+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74041f1c80 con 0x7f7404152900 2024-02-18T11:47:54.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.180+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f73fc0407f0 con 0x7f7404152900 2024-02-18T11:47:54.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.180+0000 7f73fa7fc640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e806bc70 0x7f73e806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.181+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f73fc0ad630 con 0x7f7404152900 2024-02-18T11:47:54.175 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.181+0000 7f7409d08640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e806bc70 0x7f73e806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.176 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.181+0000 7f7409d08640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e806bc70 0x7f73e806e130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7400009af0 tx=0x7f7400008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:54.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.189+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73fc077a50 con 0x7f7404152900 2024-02-18T11:47:54.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.369+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7404074ee0 con 0x7f7404152900 2024-02-18T11:47:54.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.371+0000 7f73fa7fc640 1 -- 172.21.15.116:0/3968501458 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f73fc009450 con 0x7f7404152900 2024-02-18T11:47:54.364 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:54.365 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e806bc70 msgr2=0x7f73e806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e806bc70 0x7f73e806e130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7400009af0 tx=0x7f7400008040 comp rx=0 tx=0).stop 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 msgr2=0x7f74040cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 0x7f74040cade0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009d90 tx=0x7f73fc004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 shutdown_connections 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e806bc70 0x7f73e806e130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7404152900 0x7f74040cade0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74040d0730 0x7f7404154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 --2- 172.21.15.116:0/3968501458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74040b0750 0x7f74041523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.373+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 >> 172.21.15.116:0/3968501458 conn(0x7f74040b3380 msgr2=0x7f74040cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:54.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.374+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 shutdown_connections 2024-02-18T11:47:54.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.374+0000 7f740bf6c640 1 -- 172.21.15.116:0/3968501458 wait complete. 2024-02-18T11:47:54.378 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-18T11:47:54.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.491+0000 7f6992015640 1 Processor -- start 2024-02-18T11:47:54.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.491+0000 7f6992015640 1 -- start start 2024-02-18T11:47:54.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f6992015640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 0x7f698c0b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f6992015640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c070f50 con 0x7f698c0b5880 2024-02-18T11:47:54.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f6992015640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c071230 con 0x7f698c0b4bc0 2024-02-18T11:47:54.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f6992015640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c071510 con 0x7f698c15db40 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698bfff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f698c0b5880 0x7f698c06bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:44222/0 (socket says 172.21.15.116:44222) 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698bfff640 1 -- 172.21.15.116:0/685435330 learned_addr learned my addr 172.21.15.116:0/685435330 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454844235 0 0) 0x7f698c070f50 con 0x7f698c0b5880 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6974003520 con 0x7f698c0b5880 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698b7fe640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 0x7f698c0b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105805178 0 0) 0x7f6974003520 con 0x7f698c0b5880 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 msgr2=0x7f698c0b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 0x7f698c0b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 >> v1:172.21.15.143:6789/0 conn(0x7f698c0b4bc0 legacy=0x7f698c0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0717f0 con 0x7f698c0b5880 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.492+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f69840019f0 con 0x7f698c0b5880 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 >> v1:172.21.15.116:6789/0 conn(0x7f698c0b5880 legacy=0x7f698c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 0x7f69740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6974003ee0 0x7f69740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6974004d00 0x7f6974005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c071510 con 0x7f6974004d00 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697400b9e0 con 0x7f6974003ee0 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697400bba0 con 0x7f698c15db40 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698bfff640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6974004d00 0x7f6974005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698b7fe640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 0x7f69740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698affd640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6974003ee0 0x7f69740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698bfff640 1 -- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 msgr2=0x7f69740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698bfff640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c15db40 0x7f69740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698bfff640 1 -- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6974003ee0 msgr2=0x7f69740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698bfff640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6974003ee0 0x7f69740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.493+0000 7f698bfff640 1 -- 172.21.15.116:0/685435330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974003520 con 0x7f6974004d00 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f698bfff640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6974004d00 0x7f6974005180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f698c0b5c90 tx=0x7f69740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315867b780a19775 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984034630 con 0x7f6974004d00 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698402f070 con 0x7f6974004d00 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f698a7fc640 1 -- 172.21.15.116:0/685435330 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6984034a30 con 0x7f6974004d00 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f6992015640 1 -- 172.21.15.116:0/685435330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6974004d00 msgr2=0x7f6974005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f6992015640 1 --2- 172.21.15.116:0/685435330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6974004d00 0x7f6974005180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f698c0b5c90 tx=0x7f69740057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f698bfff640 1 -- 172.21.15.116:0/685435330 reap_dead start 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f6992015640 1 -- 172.21.15.116:0/685435330 shutdown_connections 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.494+0000 7f6992015640 1 -- 172.21.15.116:0/685435330 >> 172.21.15.116:0/685435330 conn(0x7f698c06f460 msgr2=0x7f698c06f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.495+0000 7f6992015640 1 -- 172.21.15.116:0/685435330 shutdown_connections 2024-02-18T11:47:54.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.495+0000 7f6992015640 1 -- 172.21.15.116:0/685435330 wait complete. 2024-02-18T11:47:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.496+0000 7f6992015640 1 Processor -- start 2024-02-18T11:47:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.496+0000 7f6992015640 1 -- start start 2024-02-18T11:47:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c1f3950 0x7f698c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 0x7f698c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f698c1547f0 0x7f698c154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c14fe90 con 0x7f698c1f3d30 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1500e0 con 0x7f698c1547f0 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1503c0 con 0x7f698c1f3950 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 0x7f698c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 0x7f698c155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40206/0 (socket says 172.21.15.116:40206) 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 -- 172.21.15.116:0/3988517687 learned_addr learned my addr 172.21.15.116:0/3988517687 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698bfff640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f698c1547f0 0x7f698c154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698b7fe640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c1f3950 0x7f698c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 -- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c1f3950 msgr2=0x7f698c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c1f3950 0x7f698c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 -- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f698c1547f0 msgr2=0x7f698c154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f698c1547f0 0x7f698c154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 -- 172.21.15.116:0/3988517687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c1506a0 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f698affd640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 0x7f698c155a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f697c00e7d0 tx=0x7f697c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0050d0 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c005230 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c00c6c0 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c150980 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.497+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c1551b0 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.498+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c0b6170 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.499+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f697c00c820 con 0x7f698c1f3d30 2024-02-18T11:47:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.499+0000 7f6973fff640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f697806bca0 0x7f697806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:54.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.499+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f697c08a900 con 0x7f698c1f3d30 2024-02-18T11:47:54.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.499+0000 7f698b7fe640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f697806bca0 0x7f697806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:54.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.500+0000 7f698b7fe640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f697806bca0 0x7f697806e160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6980009b00 tx=0x7f6980008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:54.497 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.503+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f697c054ef0 con 0x7f698c1f3d30 2024-02-18T11:47:54.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:54.725+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f698c150ea0 con 0x7f698c1f3d30 2024-02-18T11:47:55.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.601+0000 7f6973fff640 1 -- 172.21.15.116:0/3988517687 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f698c150ea0 con 0x7f698c1f3d30 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f697806bca0 msgr2=0x7f697806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f697806bca0 0x7f697806e160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6980009b00 tx=0x7f6980008040 comp rx=0 tx=0).stop 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 msgr2=0x7f698c155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 0x7f698c155a20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f697c00e7d0 tx=0x7f697c00ec90 comp rx=0 tx=0).stop 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 shutdown_connections 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f697806bca0 0x7f697806e160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f698c1547f0 0x7f698c154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f698c1f3d30 0x7f698c155a20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 --2- 172.21.15.116:0/3988517687 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f698c1f3950 0x7f698c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.603+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 >> 172.21.15.116:0/3988517687 conn(0x7f698c06f460 msgr2=0x7f698c1584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:55.597 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.604+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 shutdown_connections 2024-02-18T11:47:55.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.604+0000 7f6992015640 1 -- 172.21.15.116:0/3988517687 wait complete. 2024-02-18T11:47:55.608 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-02-18T11:47:55.608 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-18T11:47:55.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.720+0000 7f4e54081640 1 Processor -- start 2024-02-18T11:47:55.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.720+0000 7f4e54081640 1 -- start start 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.721+0000 7f4e54081640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4e4c072db0 0x7f4e4c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.721+0000 7f4e54081640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0cdaf0 con 0x7f4e4c070930 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.721+0000 7f4e54081640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c06fa90 con 0x7f4e4c159530 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.721+0000 7f4e54081640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cdd30 con 0x7f4e4c072db0 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.721+0000 7f4e5161c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4e4c070930 0x7f4e4c0726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:44230/0 (socket says 172.21.15.116:44230) 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.721+0000 7f4e5161c640 1 -- 172.21.15.116:0/1425407452 learned_addr learned my addr 172.21.15.116:0/1425407452 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322876553 0 0) 0x7f4e3c001110 con 0x7f4e4c070930 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e38003520 con 0x7f4e4c070930 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e5261e640 1 --2- 172.21.15.116:0/1425407452 >> v2:172.21.15.143:3300/0 conn(0x7f4e4c072db0 0x7f4e4c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3384881909 0 0) 0x7f4e38003520 con 0x7f4e4c070930 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 >> v2:172.21.15.143:3300/0 conn(0x7f4e4c072db0 msgr2=0x7f4e4c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 --2- 172.21.15.116:0/1425407452 >> v2:172.21.15.143:3300/0 conn(0x7f4e4c072db0 0x7f4e4c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 >> v1:172.21.15.143:6789/0 conn(0x7f4e4c159530 legacy=0x7f4e4c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0ce010 con 0x7f4e4c070930 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f4e3c001800 con 0x7f4e4c070930 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 >> v1:172.21.15.116:6789/0 conn(0x7f4e4c070930 legacy=0x7f4e4c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e38005180 0x7f4e3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cdd30 con 0x7f4e38005180 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800c0d0 con 0x7f4e38003970 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3800c290 con 0x7f4e380047c0 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e5261e640 1 --2- 172.21.15.116:0/1425407452 >> v2:172.21.15.143:3300/0 conn(0x7f4e4c072db0 0x7f4e4c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e5261e640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e38005180 0x7f4e3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e5161c640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.722+0000 7f4e51e1d640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 -- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e380047c0 msgr2=0x7f4e38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:55.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e380047c0 0x7f4e38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 -- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e38003970 msgr2=0x7f4e38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e38003970 0x7f4e38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 -- 172.21.15.116:0/1425407452 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e38003520 con 0x7f4e38005180 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 -- 172.21.15.116:0/1425407452 reap_dead start 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e5261e640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e38005180 0x7f4e3800b980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e440098a0 tx=0x7f4e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626d37c8571f2504 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44037600 con 0x7f4e38005180 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e44037780 con 0x7f4e38005180 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.723+0000 7f4e50e1b640 1 -- 172.21.15.116:0/1425407452 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44037b30 con 0x7f4e38005180 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 -- 172.21.15.116:0/1425407452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e38005180 msgr2=0x7f4e3800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e38005180 0x7f4e3800b980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e440098a0 tx=0x7f4e380057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 -- 172.21.15.116:0/1425407452 shutdown_connections 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 --2- 172.21.15.116:0/1425407452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e38005180 0x7f4e3800b980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 -- 172.21.15.116:0/1425407452 >> 172.21.15.116:0/1425407452 conn(0x7f4e4c0b23f0 msgr2=0x7f4e4c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:55.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 -- 172.21.15.116:0/1425407452 shutdown_connections 2024-02-18T11:47:55.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.724+0000 7f4e54081640 1 -- 172.21.15.116:0/1425407452 wait complete. 2024-02-18T11:47:55.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.725+0000 7f4e54081640 1 Processor -- start 2024-02-18T11:47:55.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.725+0000 7f4e54081640 1 -- start start 2024-02-18T11:47:55.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e54081640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e38005180 0x7f4e4c14b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e54081640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 0x7f4e4c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e54081640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e4c0cbd40 0x7f4e4c0b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e54081640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cb630 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e54081640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c14b6e0 con 0x7f4e4c0cbd40 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e54081640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0c9a70 con 0x7f4e38005180 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 0x7f4e4c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 0x7f4e4c0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40218/0 (socket says 172.21.15.116:40218) 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 -- 172.21.15.116:0/964621422 learned_addr learned my addr 172.21.15.116:0/964621422 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5261e640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e4c0cbd40 0x7f4e4c0b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e51e1d640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e38005180 0x7f4e4c14b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 -- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e38005180 msgr2=0x7f4e4c14b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e38005180 0x7f4e4c14b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 -- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e4c0cbd40 msgr2=0x7f4e4c0b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e4c0cbd40 0x7f4e4c0b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:55.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 -- 172.21.15.116:0/964621422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0c9d50 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.726+0000 7f4e5161c640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 0x7f4e4c0b0cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c004fb0 tx=0x7f4e3c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.727+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00f7b0 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.727+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c00f930 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.727+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c00fce0 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.727+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c0ca030 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.727+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c0ca5f0 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.727+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c074520 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.728+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4e3c01d460 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.728+0000 7f4e427fc640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4e3006bc70 0x7f4e3006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.728+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4e3c08aab0 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.728+0000 7f4e51e1d640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4e3006bc70 0x7f4e3006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:55.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.729+0000 7f4e51e1d640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4e3006bc70 0x7f4e3006e130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0b23c0 tx=0x7f4e48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:55.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.733+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e3c055000 con 0x7f4e4c0cb8c0 2024-02-18T11:47:55.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:55.940+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4e4c0cdaf0 con 0x7f4e4c0cb8c0 2024-02-18T11:47:56.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.606+0000 7f4e427fc640 1 -- 172.21.15.116:0/964621422 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4e4c0cdaf0 con 0x7f4e4c0cb8c0 2024-02-18T11:47:56.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4e3006bc70 msgr2=0x7f4e3006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4e3006bc70 0x7f4e3006e130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0b23c0 tx=0x7f4e48009040 comp rx=0 tx=0).stop 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 msgr2=0x7f4e4c0b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 0x7f4e4c0b0cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c004fb0 tx=0x7f4e3c0074f0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 shutdown_connections 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4e3006bc70 0x7f4e3006e130 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e4c0cbd40 0x7f4e4c0b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e4c0cb8c0 0x7f4e4c0b0cf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 --2- 172.21.15.116:0/964621422 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4e38005180 0x7f4e4c14b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 >> 172.21.15.116:0/964621422 conn(0x7f4e4c0b23f0 msgr2=0x7f4e4c15cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 shutdown_connections 2024-02-18T11:47:56.602 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.608+0000 7f4e54081640 1 -- 172.21.15.116:0/964621422 wait complete. 2024-02-18T11:47:56.614 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-18T11:47:56.614 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:47:56.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.725+0000 7f722565c640 1 Processor -- start 2024-02-18T11:47:56.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.725+0000 7f722565c640 1 -- start start 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f722565c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 0x7f72200b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f722565c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f72200d1080 0x7f72200ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f722565c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72200ce780 con 0x7f7220161850 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f722565c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72200d3730 con 0x7f72200d1080 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f722565c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200cf250 con 0x7f722015dc60 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f721effd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 0x7f72200b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f721effd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 0x7f72200b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:49386/0 (socket says 172.21.15.116:49386) 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f721effd640 1 -- 172.21.15.116:0/285342900 learned_addr learned my addr 172.21.15.116:0/285342900 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:56.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.726+0000 7f721f7fe640 1 --2- 172.21.15.116:0/285342900 >> v2:172.21.15.143:3300/0 conn(0x7f72200d1080 0x7f72200ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:56.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.727+0000 7f721effd640 1 -- 172.21.15.116:0/285342900 >> v2:172.21.15.143:3300/0 conn(0x7f72200d1080 msgr2=0x7f72200ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:56.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.727+0000 7f721effd640 1 --2- 172.21.15.116:0/285342900 >> v2:172.21.15.143:3300/0 conn(0x7f72200d1080 0x7f72200ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.727+0000 7f721effd640 1 -- 172.21.15.116:0/285342900 >> v1:172.21.15.143:6789/0 conn(0x7f7220161850 legacy=0x7f72200d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.727+0000 7f721effd640 1 -- 172.21.15.116:0/285342900 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200ce9b0 con 0x7f722015dc60 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.727+0000 7f721effd640 1 --2- 172.21.15.116:0/285342900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 0x7f72200b6c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7210009870 tx=0x7f721002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace20a7b24b6b714 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.727+0000 7f721e7fc640 1 -- 172.21.15.116:0/285342900 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210030840 con 0x7f722015dc60 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f721e7fc640 1 -- 172.21.15.116:0/285342900 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72100309a0 con 0x7f722015dc60 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f721e7fc640 1 -- 172.21.15.116:0/285342900 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210030d30 con 0x7f722015dc60 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 -- 172.21.15.116:0/285342900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 msgr2=0x7f72200b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 --2- 172.21.15.116:0/285342900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 0x7f72200b6c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7210009870 tx=0x7f721002b980 comp rx=0 tx=0).stop 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 -- 172.21.15.116:0/285342900 shutdown_connections 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 --2- 172.21.15.116:0/285342900 >> v2:172.21.15.143:3300/0 conn(0x7f72200d1080 0x7f72200ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 --2- 172.21.15.116:0/285342900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f722015dc60 0x7f72200b6c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 -- 172.21.15.116:0/285342900 >> 172.21.15.116:0/285342900 conn(0x7f72200b5910 msgr2=0x7f72200b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:56.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 -- 172.21.15.116:0/285342900 shutdown_connections 2024-02-18T11:47:56.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.728+0000 7f722565c640 1 -- 172.21.15.116:0/285342900 wait complete. 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.729+0000 7f722565c640 1 Processor -- start 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.729+0000 7f722565c640 1 -- start start 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f722565c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f72200d1080 0x7f72200cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f722565c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 0x7f722014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f722565c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7220161850 0x7f7220151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f722565c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72201523f0 con 0x7f722015dc60 2024-02-18T11:47:56.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f722565c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722015f470 con 0x7f7220161850 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f722565c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722015f750 con 0x7f72200d1080 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f72167fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 0x7f722014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f72167fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 0x7f722014b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40224/0 (socket says 172.21.15.116:40224) 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f72167fc640 1 -- 172.21.15.116:0/2621013428 learned_addr learned my addr 172.21.15.116:0/2621013428 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f721effd640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f72200d1080 0x7f72200cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.730+0000 7f721f7fe640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7220161850 0x7f7220151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f72167fc640 1 -- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f72200d1080 msgr2=0x7f72200cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f72167fc640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f72200d1080 0x7f72200cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f72167fc640 1 -- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7220161850 msgr2=0x7f7220151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f72167fc640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7220161850 0x7f7220151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f72167fc640 1 -- 172.21.15.116:0/2621013428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100094d0 con 0x7f722015dc60 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f721f7fe640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7220161850 0x7f7220151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f72167fc640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 0x7f722014b0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f720800c3c0 tx=0x7f720800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72080052d0 con 0x7f722015dc60 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7208005450 con 0x7f722015dc60 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722015fa30 con 0x7f722015dc60 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7208007260 con 0x7f722015dc60 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.731+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7220152090 con 0x7f722015dc60 2024-02-18T11:47:56.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.732+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7220067840 con 0x7f722015dc60 2024-02-18T11:47:56.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.732+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f72080073c0 con 0x7f722015dc60 2024-02-18T11:47:56.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.733+0000 7f7217fff640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f720006bc70 0x7f720006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:56.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.733+0000 7f721effd640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f720006bc70 0x7f720006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:56.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.733+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7208089de0 con 0x7f722015dc60 2024-02-18T11:47:56.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.733+0000 7f721effd640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f720006bc70 0x7f720006e130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7210036000 tx=0x7f7210008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:56.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.737+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7208054330 con 0x7f722015dc60 2024-02-18T11:47:56.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.942+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f722015ff50 con 0x7f722015dc60 2024-02-18T11:47:56.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.943+0000 7f7217fff640 1 -- 172.21.15.116:0/2621013428 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f722015ff50 con 0x7f722015dc60 2024-02-18T11:47:56.936 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:56.936 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:47:56.937 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:47:56.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f720006bc70 msgr2=0x7f720006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f720006bc70 0x7f720006e130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7210036000 tx=0x7f7210008030 comp rx=0 tx=0).stop 2024-02-18T11:47:56.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 msgr2=0x7f722014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 0x7f722014b0f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f720800c3c0 tx=0x7f720800c880 comp rx=0 tx=0).stop 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 shutdown_connections 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f720006bc70 0x7f720006e130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7220161850 0x7f7220151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f722015dc60 0x7f722014b0f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 --2- 172.21.15.116:0/2621013428 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f72200d1080 0x7f72200cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 >> 172.21.15.116:0/2621013428 conn(0x7f72200b5910 msgr2=0x7f72200cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 shutdown_connections 2024-02-18T11:47:56.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:56.945+0000 7f722565c640 1 -- 172.21.15.116:0/2621013428 wait complete. 2024-02-18T11:47:56.949 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.056+0000 7fd82ec0e640 1 Processor -- start 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.056+0000 7fd82ec0e640 1 -- start start 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82ec0e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd82815aa00 0x7fd8280b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82ec0e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd8280b5110 0x7fd82815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82ec0e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd828072280 0x7fd828072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82ec0e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd828072c50 con 0x7fd828072280 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82ec0e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8280b5c20 con 0x7fd8280b5110 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82ec0e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280b45d0 con 0x7fd82815aa00 2024-02-18T11:47:57.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82d1ab640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd828072280 0x7fd828072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82d1ab640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd828072280 0x7fd828072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40236/0 (socket says 172.21.15.116:40236) 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.057+0000 7fd82d1ab640 1 -- 172.21.15.116:0/1593855160 learned_addr learned my addr 172.21.15.116:0/1593855160 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.058+0000 7fd82d1ab640 1 -- 172.21.15.116:0/1593855160 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd82815aa00 msgr2=0x7fd8280b4090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.058+0000 7fd82d1ab640 1 --2- 172.21.15.116:0/1593855160 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd82815aa00 0x7fd8280b4090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.058+0000 7fd82d1ab640 1 -- 172.21.15.116:0/1593855160 >> v2:172.21.15.143:3300/0 conn(0x7fd8280b5110 msgr2=0x7fd82815ecc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.058+0000 7fd82d1ab640 1 --2- 172.21.15.116:0/1593855160 >> v2:172.21.15.143:3300/0 conn(0x7fd8280b5110 0x7fd82815ecc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.058+0000 7fd82d1ab640 1 -- 172.21.15.116:0/1593855160 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82815e500 con 0x7fd828072280 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.058+0000 7fd82d1ab640 1 --2- 172.21.15.116:0/1593855160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd828072710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd82806dbb0 tx=0x7fd82002bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122681b5f381f4e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd8277fe640 1 -- 172.21.15.116:0/1593855160 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82002cac0 con 0x7fd828072280 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd8277fe640 1 -- 172.21.15.116:0/1593855160 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82002cc20 con 0x7fd828072280 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd8277fe640 1 -- 172.21.15.116:0/1593855160 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8200343f0 con 0x7fd828072280 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 -- 172.21.15.116:0/1593855160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 msgr2=0x7fd828072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/1593855160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd828072710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd82806dbb0 tx=0x7fd82002bf20 comp rx=0 tx=0).stop 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 -- 172.21.15.116:0/1593855160 shutdown_connections 2024-02-18T11:47:57.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/1593855160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd828072710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/1593855160 >> v2:172.21.15.143:3300/0 conn(0x7fd8280b5110 0x7fd82815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/1593855160 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd82815aa00 0x7fd8280b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 -- 172.21.15.116:0/1593855160 >> 172.21.15.116:0/1593855160 conn(0x7fd82806e280 msgr2=0x7fd82806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 -- 172.21.15.116:0/1593855160 shutdown_connections 2024-02-18T11:47:57.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.059+0000 7fd82ec0e640 1 -- 172.21.15.116:0/1593855160 wait complete. 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.060+0000 7fd82ec0e640 1 Processor -- start 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.060+0000 7fd82ec0e640 1 -- start start 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82ec0e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd82815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82ec0e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd8280b5110 0x7fd828144fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82ec0e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828159170 0x7fd828159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82ec0e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82815c880 con 0x7fd828072280 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82ec0e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828159b60 con 0x7fd828159170 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82ec0e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82815bee0 con 0x7fd8280b5110 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82c9aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd82815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82c9aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd82815cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40248/0 (socket says 172.21.15.116:40248) 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd82c9aa640 1 -- 172.21.15.116:0/842581732 learned_addr learned my addr 172.21.15.116:0/842581732 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.061+0000 7fd827fff640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd8280b5110 0x7fd828144fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82d1ab640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828159170 0x7fd828159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82c9aa640 1 -- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd8280b5110 msgr2=0x7fd828144fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82c9aa640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd8280b5110 0x7fd828144fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82c9aa640 1 -- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828159170 msgr2=0x7fd828159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82c9aa640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828159170 0x7fd828159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82c9aa640 1 -- 172.21.15.116:0/842581732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd820009b30 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82c9aa640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd82815cca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd81800b3e0 tx=0x7fd81800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011830 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8180119b0 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82815c1c0 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011d60 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.062+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8281454f0 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.063+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd828074520 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.063+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd81801a400 con 0x7fd828072280 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.064+0000 7fd8257fa640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd81406bc70 0x7fd81406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.064+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd81808ac00 con 0x7fd828072280 2024-02-18T11:47:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.064+0000 7fd827fff640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd81406bc70 0x7fd81406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.064+0000 7fd827fff640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd81406bc70 0x7fd81406e130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd82815d1e0 tx=0x7fd81c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.068+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd818091080 con 0x7fd828072280 2024-02-18T11:47:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.239+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd828152a40 con 0x7fd828072280 2024-02-18T11:47:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.241+0000 7fd8257fa640 1 -- 172.21.15.116:0/842581732 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fd818055020 con 0x7fd828072280 2024-02-18T11:47:57.234 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:57.234 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd81406bc70 msgr2=0x7fd81406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd81406bc70 0x7fd81406e130 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd82815d1e0 tx=0x7fd81c008040 comp rx=0 tx=0).stop 2024-02-18T11:47:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 msgr2=0x7fd82815cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd82815cca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd81800b3e0 tx=0x7fd81800b8a0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 shutdown_connections 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd81406bc70 0x7fd81406e130 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd828159170 0x7fd828159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd8280b5110 0x7fd828144fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 --2- 172.21.15.116:0/842581732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd828072280 0x7fd82815cca0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 >> 172.21.15.116:0/842581732 conn(0x7fd82806e280 msgr2=0x7fd8280ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 shutdown_connections 2024-02-18T11:47:57.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.243+0000 7fd82ec0e640 1 -- 172.21.15.116:0/842581732 wait complete. 2024-02-18T11:47:57.247 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-18T11:47:57.247 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:47:57.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.355+0000 7f7f00d7a640 1 Processor -- start 2024-02-18T11:47:57.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.355+0000 7f7f00d7a640 1 -- start start 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7f00d7a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7f00d7a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7efc0cfba0 0x7f7efc0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7f00d7a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0cce10 con 0x7f7efc158240 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7f00d7a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0d0030 con 0x7f7efc0cfba0 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7f00d7a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc158620 con 0x7f7efc0d1860 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7efa59c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7efc158240 0x7f7efc0d1f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:44240/0 (socket says 172.21.15.116:44240) 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7efad9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7efc0cfba0 0x7f7efc0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.356+0000 7f7efa59c640 1 -- 172.21.15.116:0/2700839203 learned_addr learned my addr 172.21.15.116:0/2700839203 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7ef9d9b640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7efad9d640 1 -- 172.21.15.116:0/2700839203 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 msgr2=0x7f7efc0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7efad9d640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc0cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7efad9d640 1 -- 172.21.15.116:0/2700839203 >> v1:172.21.15.116:6789/0 conn(0x7f7efc158240 legacy=0x7f7efc0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7efad9d640 1 -- 172.21.15.116:0/2700839203 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0cbf80 con 0x7f7efc0cfba0 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7ef959a640 1 -- 172.21.15.116:0/2700839203 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802709692 0 0) 0x7f7efc0cce10 con 0x7f7efc158240 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7ef9d9b640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.357+0000 7f7efad9d640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc0cdb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097d0 tx=0x7f7eec02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5049bdd8f2f243 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7ef959a640 1 -- 172.21.15.116:0/2700839203 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec031800 con 0x7f7efc0cfba0 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7ef959a640 1 -- 172.21.15.116:0/2700839203 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec031960 con 0x7f7efc0cfba0 2024-02-18T11:47:57.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7ef959a640 1 -- 172.21.15.116:0/2700839203 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec031cf0 con 0x7f7efc0cfba0 2024-02-18T11:47:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 -- 172.21.15.116:0/2700839203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 msgr2=0x7f7efc0cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc0cdb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097d0 tx=0x7f7eec02b8a0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 -- 172.21.15.116:0/2700839203 shutdown_connections 2024-02-18T11:47:57.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc0cdb40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/2700839203 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 -- 172.21.15.116:0/2700839203 >> 172.21.15.116:0/2700839203 conn(0x7f7efc0b3380 msgr2=0x7f7efc0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 -- 172.21.15.116:0/2700839203 shutdown_connections 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.358+0000 7f7f00d7a640 1 -- 172.21.15.116:0/2700839203 wait complete. 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 Processor -- start 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 -- start start 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc1f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7efc158240 0x7f7efc155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f7ac0 con 0x7f7efc0cfba0 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f6f40 con 0x7f7efc158240 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7f00d7a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f7220 con 0x7f7efc0d1860 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7efa59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7efa59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc1587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:40270/0 (socket says 172.21.15.116:40270) 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.360+0000 7f7efa59c640 1 -- 172.21.15.116:0/1387617658 learned_addr learned my addr 172.21.15.116:0/1387617658 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7ef9d9b640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc1f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efad9d640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7efc158240 0x7f7efc155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efa59c640 1 -- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 msgr2=0x7f7efc1f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efa59c640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc1f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efa59c640 1 -- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7efc158240 msgr2=0x7f7efc155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efa59c640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7efc158240 0x7f7efc155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efa59c640 1 -- 172.21.15.116:0/1387617658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec009400 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7efa59c640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc1587a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c7c0 tx=0x7f7ef400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400a640 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef400a7c0 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc1f7500 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400ab70 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.361+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc1f32f0 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.362+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc0c8910 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.362+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7ef400acd0 con 0x7f7efc0cfba0 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.363+0000 7f7ef2ffd640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7edc06bc70 0x7f7edc06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.363+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7ef408a360 con 0x7f7efc0cfba0 2024-02-18T11:47:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.363+0000 7f7ef9d9b640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7edc06bc70 0x7f7edc06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.363+0000 7f7ef9d9b640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7edc06bc70 0x7f7edc06e130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0ccab0 tx=0x7f7ee4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.367+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ef4054800 con 0x7f7efc0cfba0 2024-02-18T11:47:57.530 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.536+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7efc0cc4e0 con 0x7f7edc06bc70 2024-02-18T11:47:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.545+0000 7f7ef2ffd640 1 -- 172.21.15.116:0/1387617658 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f7ee4009e60 con 0x7f7edc06bc70 2024-02-18T11:47:57.542 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:57.544 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7edc06bc70 msgr2=0x7f7edc06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7edc06bc70 0x7f7edc06e130 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0ccab0 tx=0x7f7ee4008040 comp rx=0 tx=0).stop 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 msgr2=0x7f7efc1587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc1587a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c7c0 tx=0x7f7ef400cc80 comp rx=0 tx=0).stop 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 shutdown_connections 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7edc06bc70 0x7f7edc06e130 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7efc158240 0x7f7efc155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7efc0d1860 0x7f7efc1f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 --2- 172.21.15.116:0/1387617658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7efc0cfba0 0x7f7efc1587a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 >> 172.21.15.116:0/1387617658 conn(0x7f7efc0b3380 msgr2=0x7f7efc15f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 shutdown_connections 2024-02-18T11:47:57.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.548+0000 7f7f00d7a640 1 -- 172.21.15.116:0/1387617658 wait complete. 2024-02-18T11:47:57.552 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-02-18T11:47:56.294775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003067"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624689+0000","last_change":"2024-02-18T11:47:33.523482+0000","last_active":"2024-02-18T11:47:39.624689+0000","last_peered":"2024-02-18T11:47:39.624689+0000","last_clean":"2024-02-18T11:47:39.624689+0000","last_became_active":"2024-02-18T11:47:33.523129+0000","last_became_peered":"2024-02-18T11:47:33.523129+0000","last_unstale":"2024-02-18T11:47:39.624689+0000","last_undegraded":"2024-02-18T11:47:39.624689+0000","last_fullsized":"2024-02-18T11:47:39.624689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555924+0000","last_change":"2024-02-18T11:47:34.538612+0000","last_active":"2024-02-18T11:47:39.555924+0000","last_peered":"2024-02-18T11:47:39.555924+0000","last_clean":"2024-02-18T11:47:39.555924+0000","last_became_active":"2024-02-18T11:47:34.538315+0000","last_became_peered":"2024-02-18T11:47:34.538315+0000","last_unstale":"2024-02-18T11:47:39.555924+0000","last_undegraded":"2024-02-18T11:47:39.555924+0000","last_fullsized":"2024-02-18T11:47:39.555924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555044+0000","last_change":"2024-02-18T11:47:38.585025+0000","last_active":"2024-02-18T11:47:39.555044+0000","last_peered":"2024-02-18T11:47:39.555044+0000","last_clean":"2024-02-18T11:47:39.555044+0000","last_became_active":"2024-02-18T11:47:38.584669+0000","last_became_peered":"2024-02-18T11:47:38.584669+0000","last_unstale":"2024-02-18T11:47:39.555044+0000","last_undegraded":"2024-02-18T11:47:39.555044+0000","last_fullsized":"2024-02-18T11:47:39.555044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:07:55.638374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556025+0000","last_change":"2024-02-18T11:47:33.516291+0000","last_active":"2024-02-18T11:47:39.556025+0000","last_peered":"2024-02-18T11:47:39.556025+0000","last_clean":"2024-02-18T11:47:39.556025+0000","last_became_active":"2024-02-18T11:47:33.516043+0000","last_became_peered":"2024-02-18T11:47:33.516043+0000","last_unstale":"2024-02-18T11:47:39.556025+0000","last_undegraded":"2024-02-18T11:47:39.556025+0000","last_fullsized":"2024-02-18T11:47:39.556025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624545+0000","last_change":"2024-02-18T11:47:34.546759+0000","last_active":"2024-02-18T11:47:39.624545+0000","last_peered":"2024-02-18T11:47:39.624545+0000","last_clean":"2024-02-18T11:47:39.624545+0000","last_became_active":"2024-02-18T11:47:34.546282+0000","last_became_peered":"2024-02-18T11:47:34.546282+0000","last_unstale":"2024-02-18T11:47:39.624545+0000","last_undegraded":"2024-02-18T11:47:39.624545+0000","last_fullsized":"2024-02-18T11:47:39.624545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808452+0000","last_change":"2024-02-18T11:47:38.571574+0000","last_active":"2024-02-18T11:47:39.808452+0000","last_peered":"2024-02-18T11:47:39.808452+0000","last_clean":"2024-02-18T11:47:39.808452+0000","last_became_active":"2024-02-18T11:47:38.571177+0000","last_became_peered":"2024-02-18T11:47:38.571177+0000","last_unstale":"2024-02-18T11:47:39.808452+0000","last_undegraded":"2024-02-18T11:47:39.808452+0000","last_fullsized":"2024-02-18T11:47:39.808452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:29.976010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309592+0000","last_change":"2024-02-18T11:47:33.522407+0000","last_active":"2024-02-18T11:47:40.309592+0000","last_peered":"2024-02-18T11:47:40.309592+0000","last_clean":"2024-02-18T11:47:40.309592+0000","last_became_active":"2024-02-18T11:47:33.521982+0000","last_became_peered":"2024-02-18T11:47:33.521982+0000","last_unstale":"2024-02-18T11:47:40.309592+0000","last_undegraded":"2024-02-18T11:47:40.309592+0000","last_fullsized":"2024-02-18T11:47:40.309592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:54:58.132971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172176+0000","last_change":"2024-02-18T11:47:34.546074+0000","last_active":"2024-02-18T11:47:40.172176+0000","last_peered":"2024-02-18T11:47:40.172176+0000","last_clean":"2024-02-18T11:47:40.172176+0000","last_became_active":"2024-02-18T11:47:34.545447+0000","last_became_peered":"2024-02-18T11:47:34.545447+0000","last_unstale":"2024-02-18T11:47:40.172176+0000","last_undegraded":"2024-02-18T11:47:40.172176+0000","last_fullsized":"2024-02-18T11:47:40.172176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809128+0000","last_change":"2024-02-18T11:47:38.568989+0000","last_active":"2024-02-18T11:47:39.809128+0000","last_peered":"2024-02-18T11:47:39.809128+0000","last_clean":"2024-02-18T11:47:39.809128+0000","last_became_active":"2024-02-18T11:47:38.568524+0000","last_became_peered":"2024-02-18T11:47:38.568524+0000","last_unstale":"2024-02-18T11:47:39.809128+0000","last_undegraded":"2024-02-18T11:47:39.809128+0000","last_fullsized":"2024-02-18T11:47:39.809128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309327+0000","last_change":"2024-02-18T11:47:33.527284+0000","last_active":"2024-02-18T11:47:40.309327+0000","last_peered":"2024-02-18T11:47:40.309327+0000","last_clean":"2024-02-18T11:47:40.309327+0000","last_became_active":"2024-02-18T11:47:33.526669+0000","last_became_peered":"2024-02-18T11:47:33.526669+0000","last_unstale":"2024-02-18T11:47:40.309327+0000","last_undegraded":"2024-02-18T11:47:40.309327+0000","last_fullsized":"2024-02-18T11:47:40.309327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172388+0000","last_change":"2024-02-18T11:47:34.542045+0000","last_active":"2024-02-18T11:47:40.172388+0000","last_peered":"2024-02-18T11:47:40.172388+0000","last_clean":"2024-02-18T11:47:40.172388+0000","last_became_active":"2024-02-18T11:47:34.541776+0000","last_became_peered":"2024-02-18T11:47:34.541776+0000","last_unstale":"2024-02-18T11:47:40.172388+0000","last_undegraded":"2024-02-18T11:47:40.172388+0000","last_fullsized":"2024-02-18T11:47:40.172388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808568+0000","last_change":"2024-02-18T11:47:38.570454+0000","last_active":"2024-02-18T11:47:39.808568+0000","last_peered":"2024-02-18T11:47:39.808568+0000","last_clean":"2024-02-18T11:47:39.808568+0000","last_became_active":"2024-02-18T11:47:38.569640+0000","last_became_peered":"2024-02-18T11:47:38.569640+0000","last_unstale":"2024-02-18T11:47:39.808568+0000","last_undegraded":"2024-02-18T11:47:39.808568+0000","last_fullsized":"2024-02-18T11:47:39.808568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:36.793248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556080+0000","last_change":"2024-02-18T11:47:34.534695+0000","last_active":"2024-02-18T11:47:39.556080+0000","last_peered":"2024-02-18T11:47:39.556080+0000","last_clean":"2024-02-18T11:47:39.556080+0000","last_became_active":"2024-02-18T11:47:34.534368+0000","last_became_peered":"2024-02-18T11:47:34.534368+0000","last_unstale":"2024-02-18T11:47:39.556080+0000","last_undegraded":"2024-02-18T11:47:39.556080+0000","last_fullsized":"2024-02-18T11:47:39.556080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309372+0000","last_change":"2024-02-18T11:47:33.521962+0000","last_active":"2024-02-18T11:47:40.309372+0000","last_peered":"2024-02-18T11:47:40.309372+0000","last_clean":"2024-02-18T11:47:40.309372+0000","last_became_active":"2024-02-18T11:47:33.521649+0000","last_became_peered":"2024-02-18T11:47:33.521649+0000","last_unstale":"2024-02-18T11:47:40.309372+0000","last_undegraded":"2024-02-18T11:47:40.309372+0000","last_fullsized":"2024-02-18T11:47:40.309372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:37.645450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808951+0000","last_change":"2024-02-18T11:47:38.571998+0000","last_active":"2024-02-18T11:47:39.808951+0000","last_peered":"2024-02-18T11:47:39.808951+0000","last_clean":"2024-02-18T11:47:39.808951+0000","last_became_active":"2024-02-18T11:47:38.571568+0000","last_became_peered":"2024-02-18T11:47:38.571568+0000","last_unstale":"2024-02-18T11:47:39.808951+0000","last_undegraded":"2024-02-18T11:47:39.808951+0000","last_fullsized":"2024-02-18T11:47:39.808951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:47.822731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555491+0000","last_change":"2024-02-18T11:47:34.543371+0000","last_active":"2024-02-18T11:47:39.555491+0000","last_peered":"2024-02-18T11:47:39.555491+0000","last_clean":"2024-02-18T11:47:39.555491+0000","last_became_active":"2024-02-18T11:47:34.542194+0000","last_became_peered":"2024-02-18T11:47:34.542194+0000","last_unstale":"2024-02-18T11:47:39.555491+0000","last_undegraded":"2024-02-18T11:47:39.555491+0000","last_fullsized":"2024-02-18T11:47:39.555491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845296+0000","last_change":"2024-02-18T11:47:33.517942+0000","last_active":"2024-02-18T11:47:39.845296+0000","last_peered":"2024-02-18T11:47:39.845296+0000","last_clean":"2024-02-18T11:47:39.845296+0000","last_became_active":"2024-02-18T11:47:33.517664+0000","last_became_peered":"2024-02-18T11:47:33.517664+0000","last_unstale":"2024-02-18T11:47:39.845296+0000","last_undegraded":"2024-02-18T11:47:39.845296+0000","last_fullsized":"2024-02-18T11:47:39.845296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808749+0000","last_change":"2024-02-18T11:47:38.573311+0000","last_active":"2024-02-18T11:47:39.808749+0000","last_peered":"2024-02-18T11:47:39.808749+0000","last_clean":"2024-02-18T11:47:39.808749+0000","last_became_active":"2024-02-18T11:47:38.572888+0000","last_became_peered":"2024-02-18T11:47:38.572888+0000","last_unstale":"2024-02-18T11:47:39.808749+0000","last_undegraded":"2024-02-18T11:47:39.808749+0000","last_fullsized":"2024-02-18T11:47:39.808749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:51.214710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555516+0000","last_change":"2024-02-18T11:47:34.537515+0000","last_active":"2024-02-18T11:47:39.555516+0000","last_peered":"2024-02-18T11:47:39.555516+0000","last_clean":"2024-02-18T11:47:39.555516+0000","last_became_active":"2024-02-18T11:47:34.537235+0000","last_became_peered":"2024-02-18T11:47:34.537235+0000","last_unstale":"2024-02-18T11:47:39.555516+0000","last_undegraded":"2024-02-18T11:47:39.555516+0000","last_fullsized":"2024-02-18T11:47:39.555516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844871+0000","last_change":"2024-02-18T11:47:33.522206+0000","last_active":"2024-02-18T11:47:39.844871+0000","last_peered":"2024-02-18T11:47:39.844871+0000","last_clean":"2024-02-18T11:47:39.844871+0000","last_became_active":"2024-02-18T11:47:33.521878+0000","last_became_peered":"2024-02-18T11:47:33.521878+0000","last_unstale":"2024-02-18T11:47:39.844871+0000","last_undegraded":"2024-02-18T11:47:39.844871+0000","last_fullsized":"2024-02-18T11:47:39.844871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845354+0000","last_change":"2024-02-18T11:47:38.584826+0000","last_active":"2024-02-18T11:47:39.845354+0000","last_peered":"2024-02-18T11:47:39.845354+0000","last_clean":"2024-02-18T11:47:39.845354+0000","last_became_active":"2024-02-18T11:47:38.584161+0000","last_became_peered":"2024-02-18T11:47:38.584161+0000","last_unstale":"2024-02-18T11:47:39.845354+0000","last_undegraded":"2024-02-18T11:47:39.845354+0000","last_fullsized":"2024-02-18T11:47:39.845354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808871+0000","last_change":"2024-02-18T11:47:34.533256+0000","last_active":"2024-02-18T11:47:39.808871+0000","last_peered":"2024-02-18T11:47:39.808871+0000","last_clean":"2024-02-18T11:47:39.808871+0000","last_became_active":"2024-02-18T11:47:34.531682+0000","last_became_peered":"2024-02-18T11:47:34.531682+0000","last_unstale":"2024-02-18T11:47:39.808871+0000","last_undegraded":"2024-02-18T11:47:39.808871+0000","last_fullsized":"2024-02-18T11:47:39.808871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:09.297073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308926+0000","last_change":"2024-02-18T11:47:33.523406+0000","last_active":"2024-02-18T11:47:40.308926+0000","last_peered":"2024-02-18T11:47:40.308926+0000","last_clean":"2024-02-18T11:47:40.308926+0000","last_became_active":"2024-02-18T11:47:33.522929+0000","last_became_peered":"2024-02-18T11:47:33.522929+0000","last_unstale":"2024-02-18T11:47:40.308926+0000","last_undegraded":"2024-02-18T11:47:40.308926+0000","last_fullsized":"2024-02-18T11:47:40.308926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809106+0000","last_change":"2024-02-18T11:47:38.572268+0000","last_active":"2024-02-18T11:47:39.809106+0000","last_peered":"2024-02-18T11:47:39.809106+0000","last_clean":"2024-02-18T11:47:39.809106+0000","last_became_active":"2024-02-18T11:47:38.571698+0000","last_became_peered":"2024-02-18T11:47:38.571698+0000","last_unstale":"2024-02-18T11:47:39.809106+0000","last_undegraded":"2024-02-18T11:47:39.809106+0000","last_fullsized":"2024-02-18T11:47:39.809106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:59.447791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624439+0000","last_change":"2024-02-18T11:47:34.537052+0000","last_active":"2024-02-18T11:47:39.624439+0000","last_peered":"2024-02-18T11:47:39.624439+0000","last_clean":"2024-02-18T11:47:39.624439+0000","last_became_active":"2024-02-18T11:47:34.536782+0000","last_became_peered":"2024-02-18T11:47:34.536782+0000","last_unstale":"2024-02-18T11:47:39.624439+0000","last_undegraded":"2024-02-18T11:47:39.624439+0000","last_fullsized":"2024-02-18T11:47:39.624439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556919+0000","last_change":"2024-02-18T11:47:33.519918+0000","last_active":"2024-02-18T11:47:39.556919+0000","last_peered":"2024-02-18T11:47:39.556919+0000","last_clean":"2024-02-18T11:47:39.556919+0000","last_became_active":"2024-02-18T11:47:33.519651+0000","last_became_peered":"2024-02-18T11:47:33.519651+0000","last_unstale":"2024-02-18T11:47:39.556919+0000","last_undegraded":"2024-02-18T11:47:39.556919+0000","last_fullsized":"2024-02-18T11:47:39.556919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808561+0000","last_change":"2024-02-18T11:47:38.584734+0000","last_active":"2024-02-18T11:47:39.808561+0000","last_peered":"2024-02-18T11:47:39.808561+0000","last_clean":"2024-02-18T11:47:39.808561+0000","last_became_active":"2024-02-18T11:47:38.584299+0000","last_became_peered":"2024-02-18T11:47:38.584299+0000","last_unstale":"2024-02-18T11:47:39.808561+0000","last_undegraded":"2024-02-18T11:47:39.808561+0000","last_fullsized":"2024-02-18T11:47:39.808561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:59.168505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555556+0000","last_change":"2024-02-18T11:47:34.550205+0000","last_active":"2024-02-18T11:47:39.555556+0000","last_peered":"2024-02-18T11:47:39.555556+0000","last_clean":"2024-02-18T11:47:39.555556+0000","last_became_active":"2024-02-18T11:47:34.549966+0000","last_became_peered":"2024-02-18T11:47:34.549966+0000","last_unstale":"2024-02-18T11:47:39.555556+0000","last_undegraded":"2024-02-18T11:47:39.555556+0000","last_fullsized":"2024-02-18T11:47:39.555556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845365+0000","last_change":"2024-02-18T11:47:33.513489+0000","last_active":"2024-02-18T11:47:39.845365+0000","last_peered":"2024-02-18T11:47:39.845365+0000","last_clean":"2024-02-18T11:47:39.845365+0000","last_became_active":"2024-02-18T11:47:33.513181+0000","last_became_peered":"2024-02-18T11:47:33.513181+0000","last_unstale":"2024-02-18T11:47:39.845365+0000","last_undegraded":"2024-02-18T11:47:39.845365+0000","last_fullsized":"2024-02-18T11:47:39.845365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554976+0000","last_change":"2024-02-18T11:47:38.582725+0000","last_active":"2024-02-18T11:47:39.554976+0000","last_peered":"2024-02-18T11:47:39.554976+0000","last_clean":"2024-02-18T11:47:39.554976+0000","last_became_active":"2024-02-18T11:47:38.582354+0000","last_became_peered":"2024-02-18T11:47:38.582354+0000","last_unstale":"2024-02-18T11:47:39.554976+0000","last_undegraded":"2024-02-18T11:47:39.554976+0000","last_fullsized":"2024-02-18T11:47:39.554976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845190+0000","last_change":"2024-02-18T11:47:34.543899+0000","last_active":"2024-02-18T11:47:39.845190+0000","last_peered":"2024-02-18T11:47:39.845190+0000","last_clean":"2024-02-18T11:47:39.845190+0000","last_became_active":"2024-02-18T11:47:34.543627+0000","last_became_peered":"2024-02-18T11:47:34.543627+0000","last_unstale":"2024-02-18T11:47:39.845190+0000","last_undegraded":"2024-02-18T11:47:39.845190+0000","last_fullsized":"2024-02-18T11:47:39.845190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309676+0000","last_change":"2024-02-18T11:47:33.523004+0000","last_active":"2024-02-18T11:47:40.309676+0000","last_peered":"2024-02-18T11:47:40.309676+0000","last_clean":"2024-02-18T11:47:40.309676+0000","last_became_active":"2024-02-18T11:47:33.522170+0000","last_became_peered":"2024-02-18T11:47:33.522170+0000","last_unstale":"2024-02-18T11:47:40.309676+0000","last_undegraded":"2024-02-18T11:47:40.309676+0000","last_fullsized":"2024-02-18T11:47:40.309676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:12.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624145+0000","last_change":"2024-02-18T11:47:38.582313+0000","last_active":"2024-02-18T11:47:39.624145+0000","last_peered":"2024-02-18T11:47:39.624145+0000","last_clean":"2024-02-18T11:47:39.624145+0000","last_became_active":"2024-02-18T11:47:38.581439+0000","last_became_peered":"2024-02-18T11:47:38.581439+0000","last_unstale":"2024-02-18T11:47:39.624145+0000","last_undegraded":"2024-02-18T11:47:39.624145+0000","last_fullsized":"2024-02-18T11:47:39.624145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172507+0000","last_change":"2024-02-18T11:47:34.546405+0000","last_active":"2024-02-18T11:47:40.172507+0000","last_peered":"2024-02-18T11:47:40.172507+0000","last_clean":"2024-02-18T11:47:40.172507+0000","last_became_active":"2024-02-18T11:47:34.545110+0000","last_became_peered":"2024-02-18T11:47:34.545110+0000","last_unstale":"2024-02-18T11:47:40.172507+0000","last_undegraded":"2024-02-18T11:47:40.172507+0000","last_fullsized":"2024-02-18T11:47:40.172507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845298+0000","last_change":"2024-02-18T11:47:33.515238+0000","last_active":"2024-02-18T11:47:39.845298+0000","last_peered":"2024-02-18T11:47:39.845298+0000","last_clean":"2024-02-18T11:47:39.845298+0000","last_became_active":"2024-02-18T11:47:33.514792+0000","last_became_peered":"2024-02-18T11:47:33.514792+0000","last_unstale":"2024-02-18T11:47:39.845298+0000","last_undegraded":"2024-02-18T11:47:39.845298+0000","last_fullsized":"2024-02-18T11:47:39.845298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845086+0000","last_change":"2024-02-18T11:47:38.569586+0000","last_active":"2024-02-18T11:47:39.845086+0000","last_peered":"2024-02-18T11:47:39.845086+0000","last_clean":"2024-02-18T11:47:39.845086+0000","last_became_active":"2024-02-18T11:47:38.568487+0000","last_became_peered":"2024-02-18T11:47:38.568487+0000","last_unstale":"2024-02-18T11:47:39.845086+0000","last_undegraded":"2024-02-18T11:47:39.845086+0000","last_fullsized":"2024-02-18T11:47:39.845086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809233+0000","last_change":"2024-02-18T11:47:34.533351+0000","last_active":"2024-02-18T11:47:39.809233+0000","last_peered":"2024-02-18T11:47:39.809233+0000","last_clean":"2024-02-18T11:47:39.809233+0000","last_became_active":"2024-02-18T11:47:34.532667+0000","last_became_peered":"2024-02-18T11:47:34.532667+0000","last_unstale":"2024-02-18T11:47:39.809233+0000","last_undegraded":"2024-02-18T11:47:39.809233+0000","last_fullsized":"2024-02-18T11:47:39.809233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:26.230805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172223+0000","last_change":"2024-02-18T11:47:33.526859+0000","last_active":"2024-02-18T11:47:40.172223+0000","last_peered":"2024-02-18T11:47:40.172223+0000","last_clean":"2024-02-18T11:47:40.172223+0000","last_became_active":"2024-02-18T11:47:33.525839+0000","last_became_peered":"2024-02-18T11:47:33.525839+0000","last_unstale":"2024-02-18T11:47:40.172223+0000","last_undegraded":"2024-02-18T11:47:40.172223+0000","last_fullsized":"2024-02-18T11:47:40.172223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845416+0000","last_change":"2024-02-18T11:47:38.562544+0000","last_active":"2024-02-18T11:47:39.845416+0000","last_peered":"2024-02-18T11:47:39.845416+0000","last_clean":"2024-02-18T11:47:39.845416+0000","last_became_active":"2024-02-18T11:47:38.562171+0000","last_became_peered":"2024-02-18T11:47:38.562171+0000","last_unstale":"2024-02-18T11:47:39.845416+0000","last_undegraded":"2024-02-18T11:47:39.845416+0000","last_fullsized":"2024-02-18T11:47:39.845416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844931+0000","last_change":"2024-02-18T11:47:34.544098+0000","last_active":"2024-02-18T11:47:39.844931+0000","last_peered":"2024-02-18T11:47:39.844931+0000","last_clean":"2024-02-18T11:47:39.844931+0000","last_became_active":"2024-02-18T11:47:34.543279+0000","last_became_peered":"2024-02-18T11:47:34.543279+0000","last_unstale":"2024-02-18T11:47:39.844931+0000","last_undegraded":"2024-02-18T11:47:39.844931+0000","last_fullsized":"2024-02-18T11:47:39.844931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554630+0000","last_change":"2024-02-18T11:47:33.518923+0000","last_active":"2024-02-18T11:47:39.554630+0000","last_peered":"2024-02-18T11:47:39.554630+0000","last_clean":"2024-02-18T11:47:39.554630+0000","last_became_active":"2024-02-18T11:47:33.518591+0000","last_became_peered":"2024-02-18T11:47:33.518591+0000","last_unstale":"2024-02-18T11:47:39.554630+0000","last_undegraded":"2024-02-18T11:47:39.554630+0000","last_fullsized":"2024-02-18T11:47:39.554630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845451+0000","last_change":"2024-02-18T11:47:38.571577+0000","last_active":"2024-02-18T11:47:39.845451+0000","last_peered":"2024-02-18T11:47:39.845451+0000","last_clean":"2024-02-18T11:47:39.845451+0000","last_became_active":"2024-02-18T11:47:38.571165+0000","last_became_peered":"2024-02-18T11:47:38.571165+0000","last_unstale":"2024-02-18T11:47:39.845451+0000","last_undegraded":"2024-02-18T11:47:39.845451+0000","last_fullsized":"2024-02-18T11:47:39.845451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.623861+0000","last_change":"2024-02-18T11:47:34.533365+0000","last_active":"2024-02-18T11:47:39.623861+0000","last_peered":"2024-02-18T11:47:39.623861+0000","last_clean":"2024-02-18T11:47:39.623861+0000","last_became_active":"2024-02-18T11:47:34.533103+0000","last_became_peered":"2024-02-18T11:47:34.533103+0000","last_unstale":"2024-02-18T11:47:39.623861+0000","last_undegraded":"2024-02-18T11:47:39.623861+0000","last_fullsized":"2024-02-18T11:47:39.623861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:13.022192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308945+0000","last_change":"2024-02-18T11:47:33.523599+0000","last_active":"2024-02-18T11:47:40.308945+0000","last_peered":"2024-02-18T11:47:40.308945+0000","last_clean":"2024-02-18T11:47:40.308945+0000","last_became_active":"2024-02-18T11:47:33.523327+0000","last_became_peered":"2024-02-18T11:47:33.523327+0000","last_unstale":"2024-02-18T11:47:40.308945+0000","last_undegraded":"2024-02-18T11:47:40.308945+0000","last_fullsized":"2024-02-18T11:47:40.308945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624414+0000","last_change":"2024-02-18T11:47:38.581803+0000","last_active":"2024-02-18T11:47:39.624414+0000","last_peered":"2024-02-18T11:47:39.624414+0000","last_clean":"2024-02-18T11:47:39.624414+0000","last_became_active":"2024-02-18T11:47:38.581401+0000","last_became_peered":"2024-02-18T11:47:38.581401+0000","last_unstale":"2024-02-18T11:47:39.624414+0000","last_undegraded":"2024-02-18T11:47:39.624414+0000","last_fullsized":"2024-02-18T11:47:39.624414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556062+0000","last_change":"2024-02-18T11:47:34.544673+0000","last_active":"2024-02-18T11:47:39.556062+0000","last_peered":"2024-02-18T11:47:39.556062+0000","last_clean":"2024-02-18T11:47:39.556062+0000","last_became_active":"2024-02-18T11:47:34.544427+0000","last_became_peered":"2024-02-18T11:47:34.544427+0000","last_unstale":"2024-02-18T11:47:39.556062+0000","last_undegraded":"2024-02-18T11:47:39.556062+0000","last_fullsized":"2024-02-18T11:47:39.556062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172045+0000","last_change":"2024-02-18T11:47:33.522514+0000","last_active":"2024-02-18T11:47:40.172045+0000","last_peered":"2024-02-18T11:47:40.172045+0000","last_clean":"2024-02-18T11:47:40.172045+0000","last_became_active":"2024-02-18T11:47:33.522205+0000","last_became_peered":"2024-02-18T11:47:33.522205+0000","last_unstale":"2024-02-18T11:47:40.172045+0000","last_undegraded":"2024-02-18T11:47:40.172045+0000","last_fullsized":"2024-02-18T11:47:40.172045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309116+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:47:40.309116+0000","last_peered":"2024-02-18T11:47:40.309116+0000","last_clean":"2024-02-18T11:47:40.309116+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:47:40.309116+0000","last_undegraded":"2024-02-18T11:47:40.309116+0000","last_fullsized":"2024-02-18T11:47:40.309116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556628+0000","last_change":"2024-02-18T11:47:38.584484+0000","last_active":"2024-02-18T11:47:39.556628+0000","last_peered":"2024-02-18T11:47:39.556628+0000","last_clean":"2024-02-18T11:47:39.556628+0000","last_became_active":"2024-02-18T11:47:38.584089+0000","last_became_peered":"2024-02-18T11:47:38.584089+0000","last_unstale":"2024-02-18T11:47:39.556628+0000","last_undegraded":"2024-02-18T11:47:39.556628+0000","last_fullsized":"2024-02-18T11:47:39.556628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309440+0000","last_change":"2024-02-18T11:47:34.548586+0000","last_active":"2024-02-18T11:47:40.309440+0000","last_peered":"2024-02-18T11:47:40.309440+0000","last_clean":"2024-02-18T11:47:40.309440+0000","last_became_active":"2024-02-18T11:47:34.548315+0000","last_became_peered":"2024-02-18T11:47:34.548315+0000","last_unstale":"2024-02-18T11:47:40.309440+0000","last_undegraded":"2024-02-18T11:47:40.309440+0000","last_fullsized":"2024-02-18T11:47:40.309440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554890+0000","last_change":"2024-02-18T11:47:33.526959+0000","last_active":"2024-02-18T11:47:39.554890+0000","last_peered":"2024-02-18T11:47:39.554890+0000","last_clean":"2024-02-18T11:47:39.554890+0000","last_became_active":"2024-02-18T11:47:33.526632+0000","last_became_peered":"2024-02-18T11:47:33.526632+0000","last_unstale":"2024-02-18T11:47:39.554890+0000","last_undegraded":"2024-02-18T11:47:39.554890+0000","last_fullsized":"2024-02-18T11:47:39.554890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556483+0000","last_change":"2024-02-18T11:47:38.581910+0000","last_active":"2024-02-18T11:47:39.556483+0000","last_peered":"2024-02-18T11:47:39.556483+0000","last_clean":"2024-02-18T11:47:39.556483+0000","last_became_active":"2024-02-18T11:47:38.581463+0000","last_became_peered":"2024-02-18T11:47:38.581463+0000","last_unstale":"2024-02-18T11:47:39.556483+0000","last_undegraded":"2024-02-18T11:47:39.556483+0000","last_fullsized":"2024-02-18T11:47:39.556483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172330+0000","last_change":"2024-02-18T11:47:34.545863+0000","last_active":"2024-02-18T11:47:40.172330+0000","last_peered":"2024-02-18T11:47:40.172330+0000","last_clean":"2024-02-18T11:47:40.172330+0000","last_became_active":"2024-02-18T11:47:34.545142+0000","last_became_peered":"2024-02-18T11:47:34.545142+0000","last_unstale":"2024-02-18T11:47:40.172330+0000","last_undegraded":"2024-02-18T11:47:40.172330+0000","last_fullsized":"2024-02-18T11:47:40.172330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845452+0000","last_change":"2024-02-18T11:47:33.519784+0000","last_active":"2024-02-18T11:47:39.845452+0000","last_peered":"2024-02-18T11:47:39.845452+0000","last_clean":"2024-02-18T11:47:39.845452+0000","last_became_active":"2024-02-18T11:47:33.519495+0000","last_became_peered":"2024-02-18T11:47:33.519495+0000","last_unstale":"2024-02-18T11:47:39.845452+0000","last_undegraded":"2024-02-18T11:47:39.845452+0000","last_fullsized":"2024-02-18T11:47:39.845452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624233+0000","last_change":"2024-02-18T11:47:38.575385+0000","last_active":"2024-02-18T11:47:39.624233+0000","last_peered":"2024-02-18T11:47:39.624233+0000","last_clean":"2024-02-18T11:47:39.624233+0000","last_became_active":"2024-02-18T11:47:38.574974+0000","last_became_peered":"2024-02-18T11:47:38.574974+0000","last_unstale":"2024-02-18T11:47:39.624233+0000","last_undegraded":"2024-02-18T11:47:39.624233+0000","last_fullsized":"2024-02-18T11:47:39.624233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309750+0000","last_change":"2024-02-18T11:47:34.550657+0000","last_active":"2024-02-18T11:47:40.309750+0000","last_peered":"2024-02-18T11:47:40.309750+0000","last_clean":"2024-02-18T11:47:40.309750+0000","last_became_active":"2024-02-18T11:47:34.550286+0000","last_became_peered":"2024-02-18T11:47:34.550286+0000","last_unstale":"2024-02-18T11:47:40.309750+0000","last_undegraded":"2024-02-18T11:47:40.309750+0000","last_fullsized":"2024-02-18T11:47:40.309750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555138+0000","last_change":"2024-02-18T11:47:33.518941+0000","last_active":"2024-02-18T11:47:39.555138+0000","last_peered":"2024-02-18T11:47:39.555138+0000","last_clean":"2024-02-18T11:47:39.555138+0000","last_became_active":"2024-02-18T11:47:33.518644+0000","last_became_peered":"2024-02-18T11:47:33.518644+0000","last_unstale":"2024-02-18T11:47:39.555138+0000","last_undegraded":"2024-02-18T11:47:39.555138+0000","last_fullsized":"2024-02-18T11:47:39.555138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171756+0000","last_change":"2024-02-18T11:47:38.582832+0000","last_active":"2024-02-18T11:47:40.171756+0000","last_peered":"2024-02-18T11:47:40.171756+0000","last_clean":"2024-02-18T11:47:40.171756+0000","last_became_active":"2024-02-18T11:47:38.582436+0000","last_became_peered":"2024-02-18T11:47:38.582436+0000","last_unstale":"2024-02-18T11:47:40.171756+0000","last_undegraded":"2024-02-18T11:47:40.171756+0000","last_fullsized":"2024-02-18T11:47:40.171756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309643+0000","last_change":"2024-02-18T11:47:34.544607+0000","last_active":"2024-02-18T11:47:40.309643+0000","last_peered":"2024-02-18T11:47:40.309643+0000","last_clean":"2024-02-18T11:47:40.309643+0000","last_became_active":"2024-02-18T11:47:34.544257+0000","last_became_peered":"2024-02-18T11:47:34.544257+0000","last_unstale":"2024-02-18T11:47:40.309643+0000","last_undegraded":"2024-02-18T11:47:40.309643+0000","last_fullsized":"2024-02-18T11:47:40.309643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809144+0000","last_change":"2024-02-18T11:47:33.516233+0000","last_active":"2024-02-18T11:47:39.809144+0000","last_peered":"2024-02-18T11:47:39.809144+0000","last_clean":"2024-02-18T11:47:39.809144+0000","last_became_active":"2024-02-18T11:47:33.515830+0000","last_became_peered":"2024-02-18T11:47:33.515830+0000","last_unstale":"2024-02-18T11:47:39.809144+0000","last_undegraded":"2024-02-18T11:47:39.809144+0000","last_fullsized":"2024-02-18T11:47:39.809144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:43:01.501097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808439+0000","last_change":"2024-02-18T11:47:38.573381+0000","last_active":"2024-02-18T11:47:39.808439+0000","last_peered":"2024-02-18T11:47:39.808439+0000","last_clean":"2024-02-18T11:47:39.808439+0000","last_became_active":"2024-02-18T11:47:38.572824+0000","last_became_peered":"2024-02-18T11:47:38.572824+0000","last_unstale":"2024-02-18T11:47:39.808439+0000","last_undegraded":"2024-02-18T11:47:39.808439+0000","last_fullsized":"2024-02-18T11:47:39.808439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:11.073343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309074+0000","last_change":"2024-02-18T11:47:34.543813+0000","last_active":"2024-02-18T11:47:40.309074+0000","last_peered":"2024-02-18T11:47:40.309074+0000","last_clean":"2024-02-18T11:47:40.309074+0000","last_became_active":"2024-02-18T11:47:34.543520+0000","last_became_peered":"2024-02-18T11:47:34.543520+0000","last_unstale":"2024-02-18T11:47:40.309074+0000","last_undegraded":"2024-02-18T11:47:40.309074+0000","last_fullsized":"2024-02-18T11:47:40.309074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554409+0000","last_change":"2024-02-18T11:47:33.519311+0000","last_active":"2024-02-18T11:47:39.554409+0000","last_peered":"2024-02-18T11:47:39.554409+0000","last_clean":"2024-02-18T11:47:39.554409+0000","last_became_active":"2024-02-18T11:47:33.518936+0000","last_became_peered":"2024-02-18T11:47:33.518936+0000","last_unstale":"2024-02-18T11:47:39.554409+0000","last_undegraded":"2024-02-18T11:47:39.554409+0000","last_fullsized":"2024-02-18T11:47:39.554409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624339+0000","last_change":"2024-02-18T11:47:38.568770+0000","last_active":"2024-02-18T11:47:39.624339+0000","last_peered":"2024-02-18T11:47:39.624339+0000","last_clean":"2024-02-18T11:47:39.624339+0000","last_became_active":"2024-02-18T11:47:38.568394+0000","last_became_peered":"2024-02-18T11:47:38.568394+0000","last_unstale":"2024-02-18T11:47:39.624339+0000","last_undegraded":"2024-02-18T11:47:39.624339+0000","last_fullsized":"2024-02-18T11:47:39.624339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:04.112912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556127+0000","last_change":"2024-02-18T11:47:34.530166+0000","last_active":"2024-02-18T11:47:39.556127+0000","last_peered":"2024-02-18T11:47:39.556127+0000","last_clean":"2024-02-18T11:47:39.556127+0000","last_became_active":"2024-02-18T11:47:34.529875+0000","last_became_peered":"2024-02-18T11:47:34.529875+0000","last_unstale":"2024-02-18T11:47:39.556127+0000","last_undegraded":"2024-02-18T11:47:39.556127+0000","last_fullsized":"2024-02-18T11:47:39.556127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556190+0000","last_change":"2024-02-18T11:47:33.519858+0000","last_active":"2024-02-18T11:47:39.556190+0000","last_peered":"2024-02-18T11:47:39.556190+0000","last_clean":"2024-02-18T11:47:39.556190+0000","last_became_active":"2024-02-18T11:47:33.519239+0000","last_became_peered":"2024-02-18T11:47:33.519239+0000","last_unstale":"2024-02-18T11:47:39.556190+0000","last_undegraded":"2024-02-18T11:47:39.556190+0000","last_fullsized":"2024-02-18T11:47:39.556190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555696+0000","last_change":"2024-02-18T11:47:38.583646+0000","last_active":"2024-02-18T11:47:39.555696+0000","last_peered":"2024-02-18T11:47:39.555696+0000","last_clean":"2024-02-18T11:47:39.555696+0000","last_became_active":"2024-02-18T11:47:38.583283+0000","last_became_peered":"2024-02-18T11:47:38.583283+0000","last_unstale":"2024-02-18T11:47:39.555696+0000","last_undegraded":"2024-02-18T11:47:39.555696+0000","last_fullsized":"2024-02-18T11:47:39.555696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309237+0000","last_change":"2024-02-18T11:47:34.549395+0000","last_active":"2024-02-18T11:47:40.309237+0000","last_peered":"2024-02-18T11:47:40.309237+0000","last_clean":"2024-02-18T11:47:40.309237+0000","last_became_active":"2024-02-18T11:47:34.549157+0000","last_became_peered":"2024-02-18T11:47:34.549157+0000","last_unstale":"2024-02-18T11:47:40.309237+0000","last_undegraded":"2024-02-18T11:47:40.309237+0000","last_fullsized":"2024-02-18T11:47:40.309237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172210+0000","last_change":"2024-02-18T11:47:33.526266+0000","last_active":"2024-02-18T11:47:40.172210+0000","last_peered":"2024-02-18T11:47:40.172210+0000","last_clean":"2024-02-18T11:47:40.172210+0000","last_became_active":"2024-02-18T11:47:33.525885+0000","last_became_peered":"2024-02-18T11:47:33.525885+0000","last_unstale":"2024-02-18T11:47:40.172210+0000","last_undegraded":"2024-02-18T11:47:40.172210+0000","last_fullsized":"2024-02-18T11:47:40.172210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555736+0000","last_change":"2024-02-18T11:47:38.581784+0000","last_active":"2024-02-18T11:47:39.555736+0000","last_peered":"2024-02-18T11:47:39.555736+0000","last_clean":"2024-02-18T11:47:39.555736+0000","last_became_active":"2024-02-18T11:47:38.581391+0000","last_became_peered":"2024-02-18T11:47:38.581391+0000","last_unstale":"2024-02-18T11:47:39.555736+0000","last_undegraded":"2024-02-18T11:47:39.555736+0000","last_fullsized":"2024-02-18T11:47:39.555736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624104+0000","last_change":"2024-02-18T11:47:34.537306+0000","last_active":"2024-02-18T11:47:39.624104+0000","last_peered":"2024-02-18T11:47:39.624104+0000","last_clean":"2024-02-18T11:47:39.624104+0000","last_became_active":"2024-02-18T11:47:34.537048+0000","last_became_peered":"2024-02-18T11:47:34.537048+0000","last_unstale":"2024-02-18T11:47:39.624104+0000","last_undegraded":"2024-02-18T11:47:39.624104+0000","last_fullsized":"2024-02-18T11:47:39.624104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:19:55.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624290+0000","last_change":"2024-02-18T11:47:33.518377+0000","last_active":"2024-02-18T11:47:39.624290+0000","last_peered":"2024-02-18T11:47:39.624290+0000","last_clean":"2024-02-18T11:47:39.624290+0000","last_became_active":"2024-02-18T11:47:33.518105+0000","last_became_peered":"2024-02-18T11:47:33.518105+0000","last_unstale":"2024-02-18T11:47:39.624290+0000","last_undegraded":"2024-02-18T11:47:39.624290+0000","last_fullsized":"2024-02-18T11:47:39.624290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:12:42.834778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172424+0000","last_change":"2024-02-18T11:47:38.585105+0000","last_active":"2024-02-18T11:47:40.172424+0000","last_peered":"2024-02-18T11:47:40.172424+0000","last_clean":"2024-02-18T11:47:40.172424+0000","last_became_active":"2024-02-18T11:47:38.584660+0000","last_became_peered":"2024-02-18T11:47:38.584660+0000","last_unstale":"2024-02-18T11:47:40.172424+0000","last_undegraded":"2024-02-18T11:47:40.172424+0000","last_fullsized":"2024-02-18T11:47:40.172424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309312+0000","last_change":"2024-02-18T11:47:34.546219+0000","last_active":"2024-02-18T11:47:40.309312+0000","last_peered":"2024-02-18T11:47:40.309312+0000","last_clean":"2024-02-18T11:47:40.309312+0000","last_became_active":"2024-02-18T11:47:34.545961+0000","last_became_peered":"2024-02-18T11:47:34.545961+0000","last_unstale":"2024-02-18T11:47:40.309312+0000","last_undegraded":"2024-02-18T11:47:40.309312+0000","last_fullsized":"2024-02-18T11:47:40.309312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:43.783562+0000","last_change":"2024-02-18T11:47:33.511834+0000","last_active":"2024-02-18T11:47:43.783562+0000","last_peered":"2024-02-18T11:47:43.783562+0000","last_clean":"2024-02-18T11:47:43.783562+0000","last_became_active":"2024-02-18T11:47:33.511537+0000","last_became_peered":"2024-02-18T11:47:33.511537+0000","last_unstale":"2024-02-18T11:47:43.783562+0000","last_undegraded":"2024-02-18T11:47:43.783562+0000","last_fullsized":"2024-02-18T11:47:43.783562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845016+0000","last_change":"2024-02-18T11:47:38.570461+0000","last_active":"2024-02-18T11:47:39.845016+0000","last_peered":"2024-02-18T11:47:39.845016+0000","last_clean":"2024-02-18T11:47:39.845016+0000","last_became_active":"2024-02-18T11:47:38.568615+0000","last_became_peered":"2024-02-18T11:47:38.568615+0000","last_unstale":"2024-02-18T11:47:39.845016+0000","last_undegraded":"2024-02-18T11:47:39.845016+0000","last_fullsized":"2024-02-18T11:47:39.845016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808848+0000","last_change":"2024-02-18T11:47:34.541278+0000","last_active":"2024-02-18T11:47:39.808848+0000","last_peered":"2024-02-18T11:47:39.808848+0000","last_clean":"2024-02-18T11:47:39.808848+0000","last_became_active":"2024-02-18T11:47:34.540891+0000","last_became_peered":"2024-02-18T11:47:34.540891+0000","last_unstale":"2024-02-18T11:47:39.808848+0000","last_undegraded":"2024-02-18T11:47:39.808848+0000","last_fullsized":"2024-02-18T11:47:39.808848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845415+0000","last_change":"2024-02-18T11:47:33.526021+0000","last_active":"2024-02-18T11:47:39.845415+0000","last_peered":"2024-02-18T11:47:39.845415+0000","last_clean":"2024-02-18T11:47:39.845415+0000","last_became_active":"2024-02-18T11:47:33.525810+0000","last_became_peered":"2024-02-18T11:47:33.525810+0000","last_unstale":"2024-02-18T11:47:39.845415+0000","last_undegraded":"2024-02-18T11:47:39.845415+0000","last_fullsized":"2024-02-18T11:47:39.845415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809063+0000","last_change":"2024-02-18T11:47:38.568064+0000","last_active":"2024-02-18T11:47:39.809063+0000","last_peered":"2024-02-18T11:47:39.809063+0000","last_clean":"2024-02-18T11:47:39.809063+0000","last_became_active":"2024-02-18T11:47:38.567648+0000","last_became_peered":"2024-02-18T11:47:38.567648+0000","last_unstale":"2024-02-18T11:47:39.809063+0000","last_undegraded":"2024-02-18T11:47:39.809063+0000","last_fullsized":"2024-02-18T11:47:39.809063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:41.335475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624622+0000","last_change":"2024-02-18T11:47:34.546545+0000","last_active":"2024-02-18T11:47:39.624622+0000","last_peered":"2024-02-18T11:47:39.624622+0000","last_clean":"2024-02-18T11:47:39.624622+0000","last_became_active":"2024-02-18T11:47:34.545918+0000","last_became_peered":"2024-02-18T11:47:34.545918+0000","last_unstale":"2024-02-18T11:47:39.624622+0000","last_undegraded":"2024-02-18T11:47:39.624622+0000","last_fullsized":"2024-02-18T11:47:39.624622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172369+0000","last_change":"2024-02-18T11:47:33.526119+0000","last_active":"2024-02-18T11:47:40.172369+0000","last_peered":"2024-02-18T11:47:40.172369+0000","last_clean":"2024-02-18T11:47:40.172369+0000","last_became_active":"2024-02-18T11:47:33.525722+0000","last_became_peered":"2024-02-18T11:47:33.525722+0000","last_unstale":"2024-02-18T11:47:40.172369+0000","last_undegraded":"2024-02-18T11:47:40.172369+0000","last_fullsized":"2024-02-18T11:47:40.172369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555321+0000","last_change":"2024-02-18T11:47:38.569853+0000","last_active":"2024-02-18T11:47:39.555321+0000","last_peered":"2024-02-18T11:47:39.555321+0000","last_clean":"2024-02-18T11:47:39.555321+0000","last_became_active":"2024-02-18T11:47:38.569315+0000","last_became_peered":"2024-02-18T11:47:38.569315+0000","last_unstale":"2024-02-18T11:47:39.555321+0000","last_undegraded":"2024-02-18T11:47:39.555321+0000","last_fullsized":"2024-02-18T11:47:39.555321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:22:24.799952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172285+0000","last_change":"2024-02-18T11:47:34.538399+0000","last_active":"2024-02-18T11:47:40.172285+0000","last_peered":"2024-02-18T11:47:40.172285+0000","last_clean":"2024-02-18T11:47:40.172285+0000","last_became_active":"2024-02-18T11:47:34.538133+0000","last_became_peered":"2024-02-18T11:47:34.538133+0000","last_unstale":"2024-02-18T11:47:40.172285+0000","last_undegraded":"2024-02-18T11:47:40.172285+0000","last_fullsized":"2024-02-18T11:47:40.172285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556798+0000","last_change":"2024-02-18T11:47:33.526252+0000","last_active":"2024-02-18T11:47:39.556798+0000","last_peered":"2024-02-18T11:47:39.556798+0000","last_clean":"2024-02-18T11:47:39.556798+0000","last_became_active":"2024-02-18T11:47:33.525952+0000","last_became_peered":"2024-02-18T11:47:33.525952+0000","last_unstale":"2024-02-18T11:47:39.556798+0000","last_undegraded":"2024-02-18T11:47:39.556798+0000","last_fullsized":"2024-02-18T11:47:39.556798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555129+0000","last_change":"2024-02-18T11:47:38.583952+0000","last_active":"2024-02-18T11:47:39.555129+0000","last_peered":"2024-02-18T11:47:39.555129+0000","last_clean":"2024-02-18T11:47:39.555129+0000","last_became_active":"2024-02-18T11:47:38.583579+0000","last_became_peered":"2024-02-18T11:47:38.583579+0000","last_unstale":"2024-02-18T11:47:39.555129+0000","last_undegraded":"2024-02-18T11:47:39.555129+0000","last_fullsized":"2024-02-18T11:47:39.555129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:33:29.418521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624283+0000","last_change":"2024-02-18T11:47:38.583643+0000","last_active":"2024-02-18T11:47:39.624283+0000","last_peered":"2024-02-18T11:47:39.624283+0000","last_clean":"2024-02-18T11:47:39.624283+0000","last_became_active":"2024-02-18T11:47:38.583245+0000","last_became_peered":"2024-02-18T11:47:38.583245+0000","last_unstale":"2024-02-18T11:47:39.624283+0000","last_undegraded":"2024-02-18T11:47:39.624283+0000","last_fullsized":"2024-02-18T11:47:39.624283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844764+0000","last_change":"2024-02-18T11:47:34.540661+0000","last_active":"2024-02-18T11:47:39.844764+0000","last_peered":"2024-02-18T11:47:39.844764+0000","last_clean":"2024-02-18T11:47:39.844764+0000","last_became_active":"2024-02-18T11:47:34.540342+0000","last_became_peered":"2024-02-18T11:47:34.540342+0000","last_unstale":"2024-02-18T11:47:39.844764+0000","last_undegraded":"2024-02-18T11:47:39.844764+0000","last_fullsized":"2024-02-18T11:47:39.844764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171940+0000","last_change":"2024-02-18T11:47:33.526132+0000","last_active":"2024-02-18T11:47:40.171940+0000","last_peered":"2024-02-18T11:47:40.171940+0000","last_clean":"2024-02-18T11:47:40.171940+0000","last_became_active":"2024-02-18T11:47:33.525798+0000","last_became_peered":"2024-02-18T11:47:33.525798+0000","last_unstale":"2024-02-18T11:47:40.171940+0000","last_undegraded":"2024-02-18T11:47:40.171940+0000","last_fullsized":"2024-02-18T11:47:40.171940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556839+0000","last_change":"2024-02-18T11:47:38.572273+0000","last_active":"2024-02-18T11:47:39.556839+0000","last_peered":"2024-02-18T11:47:39.556839+0000","last_clean":"2024-02-18T11:47:39.556839+0000","last_became_active":"2024-02-18T11:47:38.571852+0000","last_became_peered":"2024-02-18T11:47:38.571852+0000","last_unstale":"2024-02-18T11:47:39.556839+0000","last_undegraded":"2024-02-18T11:47:39.556839+0000","last_fullsized":"2024-02-18T11:47:39.556839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555386+0000","last_change":"2024-02-18T11:47:34.546984+0000","last_active":"2024-02-18T11:47:39.555386+0000","last_peered":"2024-02-18T11:47:39.555386+0000","last_clean":"2024-02-18T11:47:39.555386+0000","last_became_active":"2024-02-18T11:47:34.546738+0000","last_became_peered":"2024-02-18T11:47:34.546738+0000","last_unstale":"2024-02-18T11:47:39.555386+0000","last_undegraded":"2024-02-18T11:47:39.555386+0000","last_fullsized":"2024-02-18T11:47:39.555386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555662+0000","last_change":"2024-02-18T11:47:33.514025+0000","last_active":"2024-02-18T11:47:39.555662+0000","last_peered":"2024-02-18T11:47:39.555662+0000","last_clean":"2024-02-18T11:47:39.555662+0000","last_became_active":"2024-02-18T11:47:33.513762+0000","last_became_peered":"2024-02-18T11:47:33.513762+0000","last_unstale":"2024-02-18T11:47:39.555662+0000","last_undegraded":"2024-02-18T11:47:39.555662+0000","last_fullsized":"2024-02-18T11:47:39.555662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556529+0000","last_change":"2024-02-18T11:47:33.521856+0000","last_active":"2024-02-18T11:47:39.556529+0000","last_peered":"2024-02-18T11:47:39.556529+0000","last_clean":"2024-02-18T11:47:39.556529+0000","last_became_active":"2024-02-18T11:47:33.521563+0000","last_became_peered":"2024-02-18T11:47:33.521563+0000","last_unstale":"2024-02-18T11:47:39.556529+0000","last_undegraded":"2024-02-18T11:47:39.556529+0000","last_fullsized":"2024-02-18T11:47:39.556529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624276+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.624276+0000","last_peered":"2024-02-18T11:47:39.624276+0000","last_clean":"2024-02-18T11:47:39.624276+0000","last_became_active":"2024-02-18T11:47:34.536344+0000","last_became_peered":"2024-02-18T11:47:34.536344+0000","last_unstale":"2024-02-18T11:47:39.624276+0000","last_undegraded":"2024-02-18T11:47:39.624276+0000","last_fullsized":"2024-02-18T11:47:39.624276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554790+0000","last_change":"2024-02-18T11:47:38.572780+0000","last_active":"2024-02-18T11:47:39.554790+0000","last_peered":"2024-02-18T11:47:39.554790+0000","last_clean":"2024-02-18T11:47:39.554790+0000","last_became_active":"2024-02-18T11:47:38.572401+0000","last_became_peered":"2024-02-18T11:47:38.572401+0000","last_unstale":"2024-02-18T11:47:39.554790+0000","last_undegraded":"2024-02-18T11:47:39.554790+0000","last_fullsized":"2024-02-18T11:47:39.554790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:06.469768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555607+0000","last_change":"2024-02-18T11:47:33.521346+0000","last_active":"2024-02-18T11:47:39.555607+0000","last_peered":"2024-02-18T11:47:39.555607+0000","last_clean":"2024-02-18T11:47:39.555607+0000","last_became_active":"2024-02-18T11:47:33.521077+0000","last_became_peered":"2024-02-18T11:47:33.521077+0000","last_unstale":"2024-02-18T11:47:39.555607+0000","last_undegraded":"2024-02-18T11:47:39.555607+0000","last_fullsized":"2024-02-18T11:47:39.555607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808655+0000","last_change":"2024-02-18T11:47:34.535139+0000","last_active":"2024-02-18T11:47:39.808655+0000","last_peered":"2024-02-18T11:47:39.808655+0000","last_clean":"2024-02-18T11:47:39.808655+0000","last_became_active":"2024-02-18T11:47:34.534871+0000","last_became_peered":"2024-02-18T11:47:34.534871+0000","last_unstale":"2024-02-18T11:47:39.808655+0000","last_undegraded":"2024-02-18T11:47:39.808655+0000","last_fullsized":"2024-02-18T11:47:39.808655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:58.805082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556009+0000","last_change":"2024-02-18T11:47:38.566494+0000","last_active":"2024-02-18T11:47:39.556009+0000","last_peered":"2024-02-18T11:47:39.556009+0000","last_clean":"2024-02-18T11:47:39.556009+0000","last_became_active":"2024-02-18T11:47:38.565672+0000","last_became_peered":"2024-02-18T11:47:38.565672+0000","last_unstale":"2024-02-18T11:47:39.556009+0000","last_undegraded":"2024-02-18T11:47:39.556009+0000","last_fullsized":"2024-02-18T11:47:39.556009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:38:29.952609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556029+0000","last_change":"2024-02-18T11:47:34.531011+0000","last_active":"2024-02-18T11:47:39.556029+0000","last_peered":"2024-02-18T11:47:39.556029+0000","last_clean":"2024-02-18T11:47:39.556029+0000","last_became_active":"2024-02-18T11:47:34.530758+0000","last_became_peered":"2024-02-18T11:47:34.530758+0000","last_unstale":"2024-02-18T11:47:39.556029+0000","last_undegraded":"2024-02-18T11:47:39.556029+0000","last_fullsized":"2024-02-18T11:47:39.556029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554518+0000","last_change":"2024-02-18T11:47:34.543755+0000","last_active":"2024-02-18T11:47:39.554518+0000","last_peered":"2024-02-18T11:47:39.554518+0000","last_clean":"2024-02-18T11:47:39.554518+0000","last_became_active":"2024-02-18T11:47:34.543500+0000","last_became_peered":"2024-02-18T11:47:34.543500+0000","last_unstale":"2024-02-18T11:47:39.554518+0000","last_undegraded":"2024-02-18T11:47:39.554518+0000","last_fullsized":"2024-02-18T11:47:39.554518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172079+0000","last_change":"2024-02-18T11:47:34.545316+0000","last_active":"2024-02-18T11:47:40.172079+0000","last_peered":"2024-02-18T11:47:40.172079+0000","last_clean":"2024-02-18T11:47:40.172079+0000","last_became_active":"2024-02-18T11:47:34.545018+0000","last_became_peered":"2024-02-18T11:47:34.545018+0000","last_unstale":"2024-02-18T11:47:40.172079+0000","last_undegraded":"2024-02-18T11:47:40.172079+0000","last_fullsized":"2024-02-18T11:47:40.172079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624183+0000","last_change":"2024-02-18T11:47:34.546010+0000","last_active":"2024-02-18T11:47:39.624183+0000","last_peered":"2024-02-18T11:47:39.624183+0000","last_clean":"2024-02-18T11:47:39.624183+0000","last_became_active":"2024-02-18T11:47:34.545247+0000","last_became_peered":"2024-02-18T11:47:34.545247+0000","last_unstale":"2024-02-18T11:47:39.624183+0000","last_undegraded":"2024-02-18T11:47:39.624183+0000","last_fullsized":"2024-02-18T11:47:39.624183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555993+0000","last_change":"2024-02-18T11:47:34.538436+0000","last_active":"2024-02-18T11:47:39.555993+0000","last_peered":"2024-02-18T11:47:39.555993+0000","last_clean":"2024-02-18T11:47:39.555993+0000","last_became_active":"2024-02-18T11:47:34.538207+0000","last_became_peered":"2024-02-18T11:47:34.538207+0000","last_unstale":"2024-02-18T11:47:39.555993+0000","last_undegraded":"2024-02-18T11:47:39.555993+0000","last_fullsized":"2024-02-18T11:47:39.555993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845187+0000","last_change":"2024-02-18T11:47:34.542741+0000","last_active":"2024-02-18T11:47:39.845187+0000","last_peered":"2024-02-18T11:47:39.845187+0000","last_clean":"2024-02-18T11:47:39.845187+0000","last_became_active":"2024-02-18T11:47:34.542467+0000","last_became_peered":"2024-02-18T11:47:34.542467+0000","last_unstale":"2024-02-18T11:47:39.845187+0000","last_undegraded":"2024-02-18T11:47:39.845187+0000","last_fullsized":"2024-02-18T11:47:39.845187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557105+0000","last_change":"2024-02-18T11:47:34.539970+0000","last_active":"2024-02-18T11:47:39.557105+0000","last_peered":"2024-02-18T11:47:39.557105+0000","last_clean":"2024-02-18T11:47:39.557105+0000","last_became_active":"2024-02-18T11:47:34.539471+0000","last_became_peered":"2024-02-18T11:47:34.539471+0000","last_unstale":"2024-02-18T11:47:39.557105+0000","last_undegraded":"2024-02-18T11:47:39.557105+0000","last_fullsized":"2024-02-18T11:47:39.557105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555437+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.555437+0000","last_peered":"2024-02-18T11:47:39.555437+0000","last_clean":"2024-02-18T11:47:39.555437+0000","last_became_active":"2024-02-18T11:47:34.536356+0000","last_became_peered":"2024-02-18T11:47:34.536356+0000","last_unstale":"2024-02-18T11:47:39.555437+0000","last_undegraded":"2024-02-18T11:47:39.555437+0000","last_fullsized":"2024-02-18T11:47:39.555437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171840+0000","last_change":"2024-02-18T11:47:34.542717+0000","last_active":"2024-02-18T11:47:40.171840+0000","last_peered":"2024-02-18T11:47:40.171840+0000","last_clean":"2024-02-18T11:47:40.171840+0000","last_became_active":"2024-02-18T11:47:34.542470+0000","last_became_peered":"2024-02-18T11:47:34.542470+0000","last_unstale":"2024-02-18T11:47:40.171840+0000","last_undegraded":"2024-02-18T11:47:40.171840+0000","last_fullsized":"2024-02-18T11:47:40.171840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555193+0000","last_change":"2024-02-18T11:47:34.537727+0000","last_active":"2024-02-18T11:47:39.555193+0000","last_peered":"2024-02-18T11:47:39.555193+0000","last_clean":"2024-02-18T11:47:39.555193+0000","last_became_active":"2024-02-18T11:47:34.536902+0000","last_became_peered":"2024-02-18T11:47:34.536902+0000","last_unstale":"2024-02-18T11:47:39.555193+0000","last_undegraded":"2024-02-18T11:47:39.555193+0000","last_fullsized":"2024-02-18T11:47:39.555193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172026+0000","last_change":"2024-02-18T11:47:34.549313+0000","last_active":"2024-02-18T11:47:40.172026+0000","last_peered":"2024-02-18T11:47:40.172026+0000","last_clean":"2024-02-18T11:47:40.172026+0000","last_became_active":"2024-02-18T11:47:34.549010+0000","last_became_peered":"2024-02-18T11:47:34.549010+0000","last_unstale":"2024-02-18T11:47:40.172026+0000","last_undegraded":"2024-02-18T11:47:40.172026+0000","last_fullsized":"2024-02-18T11:47:40.172026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624061+0000","last_change":"2024-02-18T11:47:34.546238+0000","last_active":"2024-02-18T11:47:39.624061+0000","last_peered":"2024-02-18T11:47:39.624061+0000","last_clean":"2024-02-18T11:47:39.624061+0000","last_became_active":"2024-02-18T11:47:34.545634+0000","last_became_peered":"2024-02-18T11:47:34.545634+0000","last_unstale":"2024-02-18T11:47:39.624061+0000","last_undegraded":"2024-02-18T11:47:39.624061+0000","last_fullsized":"2024-02-18T11:47:39.624061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:18:21.067312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556482+0000","last_change":"2024-02-18T11:47:34.534576+0000","last_active":"2024-02-18T11:47:39.556482+0000","last_peered":"2024-02-18T11:47:39.556482+0000","last_clean":"2024-02-18T11:47:39.556482+0000","last_became_active":"2024-02-18T11:47:34.534232+0000","last_became_peered":"2024-02-18T11:47:34.534232+0000","last_unstale":"2024-02-18T11:47:39.556482+0000","last_undegraded":"2024-02-18T11:47:39.556482+0000","last_fullsized":"2024-02-18T11:47:39.556482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309588+0000","last_change":"2024-02-18T11:47:34.546750+0000","last_active":"2024-02-18T11:47:40.309588+0000","last_peered":"2024-02-18T11:47:40.309588+0000","last_clean":"2024-02-18T11:47:40.309588+0000","last_became_active":"2024-02-18T11:47:34.546507+0000","last_became_peered":"2024-02-18T11:47:34.546507+0000","last_unstale":"2024-02-18T11:47:40.309588+0000","last_undegraded":"2024-02-18T11:47:40.309588+0000","last_fullsized":"2024-02-18T11:47:40.309588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555043+0000","last_change":"2024-02-18T11:47:34.542028+0000","last_active":"2024-02-18T11:47:39.555043+0000","last_peered":"2024-02-18T11:47:39.555043+0000","last_clean":"2024-02-18T11:47:39.555043+0000","last_became_active":"2024-02-18T11:47:34.541653+0000","last_became_peered":"2024-02-18T11:47:34.541653+0000","last_unstale":"2024-02-18T11:47:39.555043+0000","last_undegraded":"2024-02-18T11:47:39.555043+0000","last_fullsized":"2024-02-18T11:47:39.555043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:38.652128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172499+0000","last_change":"2024-02-18T11:47:34.549989+0000","last_active":"2024-02-18T11:47:40.172499+0000","last_peered":"2024-02-18T11:47:40.172499+0000","last_clean":"2024-02-18T11:47:40.172499+0000","last_became_active":"2024-02-18T11:47:34.549661+0000","last_became_peered":"2024-02-18T11:47:34.549661+0000","last_unstale":"2024-02-18T11:47:40.172499+0000","last_undegraded":"2024-02-18T11:47:40.172499+0000","last_fullsized":"2024-02-18T11:47:40.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171732+0000","last_change":"2024-02-18T11:47:34.547081+0000","last_active":"2024-02-18T11:47:40.171732+0000","last_peered":"2024-02-18T11:47:40.171732+0000","last_clean":"2024-02-18T11:47:40.171732+0000","last_became_active":"2024-02-18T11:47:34.546812+0000","last_became_peered":"2024-02-18T11:47:34.546812+0000","last_unstale":"2024-02-18T11:47:40.171732+0000","last_undegraded":"2024-02-18T11:47:40.171732+0000","last_fullsized":"2024-02-18T11:47:40.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844848+0000","last_change":"2024-02-18T11:47:34.534907+0000","last_active":"2024-02-18T11:47:39.844848+0000","last_peered":"2024-02-18T11:47:39.844848+0000","last_clean":"2024-02-18T11:47:39.844848+0000","last_became_active":"2024-02-18T11:47:34.534661+0000","last_became_peered":"2024-02-18T11:47:34.534661+0000","last_unstale":"2024-02-18T11:47:39.844848+0000","last_undegraded":"2024-02-18T11:47:39.844848+0000","last_fullsized":"2024-02-18T11:47:39.844848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624050+0000","last_change":"2024-02-18T11:47:34.537888+0000","last_active":"2024-02-18T11:47:39.624050+0000","last_peered":"2024-02-18T11:47:39.624050+0000","last_clean":"2024-02-18T11:47:39.624050+0000","last_became_active":"2024-02-18T11:47:34.537647+0000","last_became_peered":"2024-02-18T11:47:34.537647+0000","last_unstale":"2024-02-18T11:47:39.624050+0000","last_undegraded":"2024-02-18T11:47:39.624050+0000","last_fullsized":"2024-02-18T11:47:39.624050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:02.426785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555401+0000","last_change":"2024-02-18T11:47:34.541254+0000","last_active":"2024-02-18T11:47:39.555401+0000","last_peered":"2024-02-18T11:47:39.555401+0000","last_clean":"2024-02-18T11:47:39.555401+0000","last_became_active":"2024-02-18T11:47:34.540955+0000","last_became_peered":"2024-02-18T11:47:34.540955+0000","last_unstale":"2024-02-18T11:47:39.555401+0000","last_undegraded":"2024-02-18T11:47:39.555401+0000","last_fullsized":"2024-02-18T11:47:39.555401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808837+0000","last_change":"2024-02-18T11:47:34.531217+0000","last_active":"2024-02-18T11:47:39.808837+0000","last_peered":"2024-02-18T11:47:39.808837+0000","last_clean":"2024-02-18T11:47:39.808837+0000","last_became_active":"2024-02-18T11:47:34.530120+0000","last_became_peered":"2024-02-18T11:47:34.530120+0000","last_unstale":"2024-02-18T11:47:39.808837+0000","last_undegraded":"2024-02-18T11:47:39.808837+0000","last_fullsized":"2024-02-18T11:47:39.808837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:43.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556645+0000","last_change":"2024-02-18T11:47:34.544075+0000","last_active":"2024-02-18T11:47:39.556645+0000","last_peered":"2024-02-18T11:47:39.556645+0000","last_clean":"2024-02-18T11:47:39.556645+0000","last_became_active":"2024-02-18T11:47:34.543761+0000","last_became_peered":"2024-02-18T11:47:34.543761+0000","last_unstale":"2024-02-18T11:47:39.556645+0000","last_undegraded":"2024-02-18T11:47:39.556645+0000","last_fullsized":"2024-02-18T11:47:39.556645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557031+0000","last_change":"2024-02-18T11:47:34.546387+0000","last_active":"2024-02-18T11:47:39.557031+0000","last_peered":"2024-02-18T11:47:39.557031+0000","last_clean":"2024-02-18T11:47:39.557031+0000","last_became_active":"2024-02-18T11:47:34.546115+0000","last_became_peered":"2024-02-18T11:47:34.546115+0000","last_unstale":"2024-02-18T11:47:39.557031+0000","last_undegraded":"2024-02-18T11:47:39.557031+0000","last_fullsized":"2024-02-18T11:47:39.557031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309697+0000","last_change":"2024-02-18T11:47:34.552880+0000","last_active":"2024-02-18T11:47:40.309697+0000","last_peered":"2024-02-18T11:47:40.309697+0000","last_clean":"2024-02-18T11:47:40.309697+0000","last_became_active":"2024-02-18T11:47:34.552530+0000","last_became_peered":"2024-02-18T11:47:34.552530+0000","last_unstale":"2024-02-18T11:47:40.309697+0000","last_undegraded":"2024-02-18T11:47:40.309697+0000","last_fullsized":"2024-02-18T11:47:40.309697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555106+0000","last_change":"2024-02-18T11:47:34.545727+0000","last_active":"2024-02-18T11:47:39.555106+0000","last_peered":"2024-02-18T11:47:39.555106+0000","last_clean":"2024-02-18T11:47:39.555106+0000","last_became_active":"2024-02-18T11:47:34.545475+0000","last_became_peered":"2024-02-18T11:47:34.545475+0000","last_unstale":"2024-02-18T11:47:39.555106+0000","last_undegraded":"2024-02-18T11:47:39.555106+0000","last_fullsized":"2024-02-18T11:47:39.555106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309012+0000","last_change":"2024-02-18T11:47:34.547696+0000","last_active":"2024-02-18T11:47:40.309012+0000","last_peered":"2024-02-18T11:47:40.309012+0000","last_clean":"2024-02-18T11:47:40.309012+0000","last_became_active":"2024-02-18T11:47:34.547448+0000","last_became_peered":"2024-02-18T11:47:34.547448+0000","last_unstale":"2024-02-18T11:47:40.309012+0000","last_undegraded":"2024-02-18T11:47:40.309012+0000","last_fullsized":"2024-02-18T11:47:40.309012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171835+0000","last_change":"2024-02-18T11:47:34.548053+0000","last_active":"2024-02-18T11:47:40.171835+0000","last_peered":"2024-02-18T11:47:40.171835+0000","last_clean":"2024-02-18T11:47:40.171835+0000","last_became_active":"2024-02-18T11:47:34.547181+0000","last_became_peered":"2024-02-18T11:47:34.547181+0000","last_unstale":"2024-02-18T11:47:40.171835+0000","last_undegraded":"2024-02-18T11:47:40.171835+0000","last_fullsized":"2024-02-18T11:47:40.171835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309134+0000","last_change":"2024-02-18T11:47:34.549242+0000","last_active":"2024-02-18T11:47:40.309134+0000","last_peered":"2024-02-18T11:47:40.309134+0000","last_clean":"2024-02-18T11:47:40.309134+0000","last_became_active":"2024-02-18T11:47:34.548872+0000","last_became_peered":"2024-02-18T11:47:34.548872+0000","last_unstale":"2024-02-18T11:47:40.309134+0000","last_undegraded":"2024-02-18T11:47:40.309134+0000","last_fullsized":"2024-02-18T11:47:40.309134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172083+0000","last_change":"2024-02-18T11:47:34.546466+0000","last_active":"2024-02-18T11:47:40.172083+0000","last_peered":"2024-02-18T11:47:40.172083+0000","last_clean":"2024-02-18T11:47:40.172083+0000","last_became_active":"2024-02-18T11:47:34.546218+0000","last_became_peered":"2024-02-18T11:47:34.546218+0000","last_unstale":"2024-02-18T11:47:40.172083+0000","last_undegraded":"2024-02-18T11:47:40.172083+0000","last_fullsized":"2024-02-18T11:47:40.172083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624360+0000","last_change":"2024-02-18T11:47:34.530060+0000","last_active":"2024-02-18T11:47:39.624360+0000","last_peered":"2024-02-18T11:47:39.624360+0000","last_clean":"2024-02-18T11:47:39.624360+0000","last_became_active":"2024-02-18T11:47:34.529799+0000","last_became_peered":"2024-02-18T11:47:34.529799+0000","last_unstale":"2024-02-18T11:47:39.624360+0000","last_undegraded":"2024-02-18T11:47:39.624360+0000","last_fullsized":"2024-02-18T11:47:39.624360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845225+0000","last_change":"2024-02-18T11:47:34.547273+0000","last_active":"2024-02-18T11:47:39.845225+0000","last_peered":"2024-02-18T11:47:39.845225+0000","last_clean":"2024-02-18T11:47:39.845225+0000","last_became_active":"2024-02-18T11:47:34.546984+0000","last_became_peered":"2024-02-18T11:47:34.546984+0000","last_unstale":"2024-02-18T11:47:39.845225+0000","last_undegraded":"2024-02-18T11:47:39.845225+0000","last_fullsized":"2024-02-18T11:47:39.845225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309602+0000","last_change":"2024-02-18T11:47:34.547456+0000","last_active":"2024-02-18T11:47:40.309602+0000","last_peered":"2024-02-18T11:47:40.309602+0000","last_clean":"2024-02-18T11:47:40.309602+0000","last_became_active":"2024-02-18T11:47:34.547193+0000","last_became_peered":"2024-02-18T11:47:34.547193+0000","last_unstale":"2024-02-18T11:47:40.309602+0000","last_undegraded":"2024-02-18T11:47:40.309602+0000","last_fullsized":"2024-02-18T11:47:40.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:47:57.561 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:47:57.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.700+0000 7fc6a8815640 1 Processor -- start 2024-02-18T11:47:57.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.700+0000 7fc6a8815640 1 -- start start 2024-02-18T11:47:57.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a8815640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a4161fe0 0x7fc6a40cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a8815640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6a40b3680 0x7fc6a40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a8815640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a41623c0 con 0x7fc6a40b3b00 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a8815640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a40cd9f0 con 0x7fc6a40b3680 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a8815640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40723b0 con 0x7fc6a4161fe0 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6a40b3680 0x7fc6a40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6a40b3680 0x7fc6a40b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57158/0 (socket says 172.21.15.116:57158) 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2023923107 learned_addr learned my addr 172.21.15.116:0/2023923107 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a259c640 1 --2- 172.21.15.116:0/2023923107 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a4161fe0 0x7fc6a40cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2023923107 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a4161fe0 msgr2=0x7fc6a40cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 --2- 172.21.15.116:0/2023923107 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a4161fe0 0x7fc6a40cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2023923107 >> v1:172.21.15.143:6789/0 conn(0x7fc6a40b3b00 legacy=0x7fc6a4165c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.701+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2023923107 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4072690 con 0x7fc6a40b3680 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.702+0000 7fc6a1d9b640 1 --2- 172.21.15.116:0/2023923107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3680 0x7fc6a40b4190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc698009870 tx=0x7fc69802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0f543ea08d6341 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.702+0000 7fc6a159a640 1 -- 172.21.15.116:0/2023923107 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6980317c0 con 0x7fc6a40b3680 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.702+0000 7fc6a159a640 1 -- 172.21.15.116:0/2023923107 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc698031920 con 0x7fc6a40b3680 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.702+0000 7fc6a159a640 1 -- 172.21.15.116:0/2023923107 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698031cb0 con 0x7fc6a40b3680 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 -- 172.21.15.116:0/2023923107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3680 msgr2=0x7fc6a40b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2023923107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3680 0x7fc6a40b4190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc698009870 tx=0x7fc69802b860 comp rx=0 tx=0).stop 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 -- 172.21.15.116:0/2023923107 shutdown_connections 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2023923107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3680 0x7fc6a40b4190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2023923107 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a4161fe0 0x7fc6a40cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 -- 172.21.15.116:0/2023923107 >> 172.21.15.116:0/2023923107 conn(0x7fc6a406e280 msgr2=0x7fc6a406e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 -- 172.21.15.116:0/2023923107 shutdown_connections 2024-02-18T11:47:57.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.703+0000 7fc6a8815640 1 -- 172.21.15.116:0/2023923107 wait complete. 2024-02-18T11:47:57.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.704+0000 7fc6a8815640 1 Processor -- start 2024-02-18T11:47:57.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.704+0000 7fc6a8815640 1 -- start start 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a8815640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a40b3680 0x7fc6a4144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a8815640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 0x7fc6a4157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a8815640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6a4161fe0 0x7fc6a4158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a8815640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a41636d0 con 0x7fc6a40b3b00 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a8815640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a415acc0 con 0x7fc6a4161fe0 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a8815640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a415afa0 con 0x7fc6a40b3680 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 0x7fc6a4157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 0x7fc6a4157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57160/0 (socket says 172.21.15.116:57160) 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2353409611 learned_addr learned my addr 172.21.15.116:0/2353409611 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a259c640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a40b3680 0x7fc6a4144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a2d9d640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6a4161fe0 0x7fc6a4158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a40b3680 msgr2=0x7fc6a4144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a40b3680 0x7fc6a4144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6a4161fe0 msgr2=0x7fc6a4158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6a4161fe0 0x7fc6a4158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.705+0000 7fc6a1d9b640 1 -- 172.21.15.116:0/2353409611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980094d0 con 0x7fc6a40b3b00 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.706+0000 7fc6a1d9b640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 0x7fc6a4157dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc69802b8a0 tx=0x7fc6980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.706+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698032860 con 0x7fc6a40b3b00 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.706+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc698032a00 con 0x7fc6a40b3b00 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.706+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a40b0160 con 0x7fc6a40b3b00 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.706+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698032db0 con 0x7fc6a40b3b00 2024-02-18T11:47:57.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.706+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a4163c00 con 0x7fc6a40b3b00 2024-02-18T11:47:57.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.707+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a4074520 con 0x7fc6a40b3b00 2024-02-18T11:47:57.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.707+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc69804e450 con 0x7fc6a40b3b00 2024-02-18T11:47:57.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.707+0000 7fc692ffd640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc68406bc70 0x7fc68406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:57.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.707+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6980acbf0 con 0x7fc6a40b3b00 2024-02-18T11:47:57.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.708+0000 7fc6a259c640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc68406bc70 0x7fc68406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:57.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.708+0000 7fc6a259c640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc68406bc70 0x7fc68406e130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc6a41fb3d0 tx=0x7fc694009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:57.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.712+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc698077200 con 0x7fc6a40b3b00 2024-02-18T11:47:57.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.878+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6a40affe0 con 0x7fc68406bc70 2024-02-18T11:47:57.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.886+0000 7fc692ffd640 1 -- 172.21.15.116:0/2353409611 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fc694004700 con 0x7fc68406bc70 2024-02-18T11:47:57.884 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:57.886 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.889+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc68406bc70 msgr2=0x7fc68406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.889+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc68406bc70 0x7fc68406e130 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc6a41fb3d0 tx=0x7fc694009830 comp rx=0 tx=0).stop 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.889+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 msgr2=0x7fc6a4157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.889+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 0x7fc6a4157dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc69802b8a0 tx=0x7fc6980046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.889+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 shutdown_connections 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc68406bc70 0x7fc68406e130 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6a4161fe0 0x7fc6a4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6a40b3b00 0x7fc6a4157dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 --2- 172.21.15.116:0/2353409611 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc6a40b3680 0x7fc6a4144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 >> 172.21.15.116:0/2353409611 conn(0x7fc6a406e280 msgr2=0x7fc6a40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:57.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 shutdown_connections 2024-02-18T11:47:57.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:57.890+0000 7fc6a8815640 1 -- 172.21.15.116:0/2353409611 wait complete. 2024-02-18T11:47:57.894 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-02-18T11:47:56.294775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003067"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624689+0000","last_change":"2024-02-18T11:47:33.523482+0000","last_active":"2024-02-18T11:47:39.624689+0000","last_peered":"2024-02-18T11:47:39.624689+0000","last_clean":"2024-02-18T11:47:39.624689+0000","last_became_active":"2024-02-18T11:47:33.523129+0000","last_became_peered":"2024-02-18T11:47:33.523129+0000","last_unstale":"2024-02-18T11:47:39.624689+0000","last_undegraded":"2024-02-18T11:47:39.624689+0000","last_fullsized":"2024-02-18T11:47:39.624689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555924+0000","last_change":"2024-02-18T11:47:34.538612+0000","last_active":"2024-02-18T11:47:39.555924+0000","last_peered":"2024-02-18T11:47:39.555924+0000","last_clean":"2024-02-18T11:47:39.555924+0000","last_became_active":"2024-02-18T11:47:34.538315+0000","last_became_peered":"2024-02-18T11:47:34.538315+0000","last_unstale":"2024-02-18T11:47:39.555924+0000","last_undegraded":"2024-02-18T11:47:39.555924+0000","last_fullsized":"2024-02-18T11:47:39.555924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555044+0000","last_change":"2024-02-18T11:47:38.585025+0000","last_active":"2024-02-18T11:47:39.555044+0000","last_peered":"2024-02-18T11:47:39.555044+0000","last_clean":"2024-02-18T11:47:39.555044+0000","last_became_active":"2024-02-18T11:47:38.584669+0000","last_became_peered":"2024-02-18T11:47:38.584669+0000","last_unstale":"2024-02-18T11:47:39.555044+0000","last_undegraded":"2024-02-18T11:47:39.555044+0000","last_fullsized":"2024-02-18T11:47:39.555044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:07:55.638374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556025+0000","last_change":"2024-02-18T11:47:33.516291+0000","last_active":"2024-02-18T11:47:39.556025+0000","last_peered":"2024-02-18T11:47:39.556025+0000","last_clean":"2024-02-18T11:47:39.556025+0000","last_became_active":"2024-02-18T11:47:33.516043+0000","last_became_peered":"2024-02-18T11:47:33.516043+0000","last_unstale":"2024-02-18T11:47:39.556025+0000","last_undegraded":"2024-02-18T11:47:39.556025+0000","last_fullsized":"2024-02-18T11:47:39.556025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624545+0000","last_change":"2024-02-18T11:47:34.546759+0000","last_active":"2024-02-18T11:47:39.624545+0000","last_peered":"2024-02-18T11:47:39.624545+0000","last_clean":"2024-02-18T11:47:39.624545+0000","last_became_active":"2024-02-18T11:47:34.546282+0000","last_became_peered":"2024-02-18T11:47:34.546282+0000","last_unstale":"2024-02-18T11:47:39.624545+0000","last_undegraded":"2024-02-18T11:47:39.624545+0000","last_fullsized":"2024-02-18T11:47:39.624545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808452+0000","last_change":"2024-02-18T11:47:38.571574+0000","last_active":"2024-02-18T11:47:39.808452+0000","last_peered":"2024-02-18T11:47:39.808452+0000","last_clean":"2024-02-18T11:47:39.808452+0000","last_became_active":"2024-02-18T11:47:38.571177+0000","last_became_peered":"2024-02-18T11:47:38.571177+0000","last_unstale":"2024-02-18T11:47:39.808452+0000","last_undegraded":"2024-02-18T11:47:39.808452+0000","last_fullsized":"2024-02-18T11:47:39.808452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:29.976010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309592+0000","last_change":"2024-02-18T11:47:33.522407+0000","last_active":"2024-02-18T11:47:40.309592+0000","last_peered":"2024-02-18T11:47:40.309592+0000","last_clean":"2024-02-18T11:47:40.309592+0000","last_became_active":"2024-02-18T11:47:33.521982+0000","last_became_peered":"2024-02-18T11:47:33.521982+0000","last_unstale":"2024-02-18T11:47:40.309592+0000","last_undegraded":"2024-02-18T11:47:40.309592+0000","last_fullsized":"2024-02-18T11:47:40.309592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:54:58.132971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172176+0000","last_change":"2024-02-18T11:47:34.546074+0000","last_active":"2024-02-18T11:47:40.172176+0000","last_peered":"2024-02-18T11:47:40.172176+0000","last_clean":"2024-02-18T11:47:40.172176+0000","last_became_active":"2024-02-18T11:47:34.545447+0000","last_became_peered":"2024-02-18T11:47:34.545447+0000","last_unstale":"2024-02-18T11:47:40.172176+0000","last_undegraded":"2024-02-18T11:47:40.172176+0000","last_fullsized":"2024-02-18T11:47:40.172176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809128+0000","last_change":"2024-02-18T11:47:38.568989+0000","last_active":"2024-02-18T11:47:39.809128+0000","last_peered":"2024-02-18T11:47:39.809128+0000","last_clean":"2024-02-18T11:47:39.809128+0000","last_became_active":"2024-02-18T11:47:38.568524+0000","last_became_peered":"2024-02-18T11:47:38.568524+0000","last_unstale":"2024-02-18T11:47:39.809128+0000","last_undegraded":"2024-02-18T11:47:39.809128+0000","last_fullsized":"2024-02-18T11:47:39.809128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309327+0000","last_change":"2024-02-18T11:47:33.527284+0000","last_active":"2024-02-18T11:47:40.309327+0000","last_peered":"2024-02-18T11:47:40.309327+0000","last_clean":"2024-02-18T11:47:40.309327+0000","last_became_active":"2024-02-18T11:47:33.526669+0000","last_became_peered":"2024-02-18T11:47:33.526669+0000","last_unstale":"2024-02-18T11:47:40.309327+0000","last_undegraded":"2024-02-18T11:47:40.309327+0000","last_fullsized":"2024-02-18T11:47:40.309327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172388+0000","last_change":"2024-02-18T11:47:34.542045+0000","last_active":"2024-02-18T11:47:40.172388+0000","last_peered":"2024-02-18T11:47:40.172388+0000","last_clean":"2024-02-18T11:47:40.172388+0000","last_became_active":"2024-02-18T11:47:34.541776+0000","last_became_peered":"2024-02-18T11:47:34.541776+0000","last_unstale":"2024-02-18T11:47:40.172388+0000","last_undegraded":"2024-02-18T11:47:40.172388+0000","last_fullsized":"2024-02-18T11:47:40.172388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808568+0000","last_change":"2024-02-18T11:47:38.570454+0000","last_active":"2024-02-18T11:47:39.808568+0000","last_peered":"2024-02-18T11:47:39.808568+0000","last_clean":"2024-02-18T11:47:39.808568+0000","last_became_active":"2024-02-18T11:47:38.569640+0000","last_became_peered":"2024-02-18T11:47:38.569640+0000","last_unstale":"2024-02-18T11:47:39.808568+0000","last_undegraded":"2024-02-18T11:47:39.808568+0000","last_fullsized":"2024-02-18T11:47:39.808568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:36.793248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556080+0000","last_change":"2024-02-18T11:47:34.534695+0000","last_active":"2024-02-18T11:47:39.556080+0000","last_peered":"2024-02-18T11:47:39.556080+0000","last_clean":"2024-02-18T11:47:39.556080+0000","last_became_active":"2024-02-18T11:47:34.534368+0000","last_became_peered":"2024-02-18T11:47:34.534368+0000","last_unstale":"2024-02-18T11:47:39.556080+0000","last_undegraded":"2024-02-18T11:47:39.556080+0000","last_fullsized":"2024-02-18T11:47:39.556080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309372+0000","last_change":"2024-02-18T11:47:33.521962+0000","last_active":"2024-02-18T11:47:40.309372+0000","last_peered":"2024-02-18T11:47:40.309372+0000","last_clean":"2024-02-18T11:47:40.309372+0000","last_became_active":"2024-02-18T11:47:33.521649+0000","last_became_peered":"2024-02-18T11:47:33.521649+0000","last_unstale":"2024-02-18T11:47:40.309372+0000","last_undegraded":"2024-02-18T11:47:40.309372+0000","last_fullsized":"2024-02-18T11:47:40.309372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:37.645450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808951+0000","last_change":"2024-02-18T11:47:38.571998+0000","last_active":"2024-02-18T11:47:39.808951+0000","last_peered":"2024-02-18T11:47:39.808951+0000","last_clean":"2024-02-18T11:47:39.808951+0000","last_became_active":"2024-02-18T11:47:38.571568+0000","last_became_peered":"2024-02-18T11:47:38.571568+0000","last_unstale":"2024-02-18T11:47:39.808951+0000","last_undegraded":"2024-02-18T11:47:39.808951+0000","last_fullsized":"2024-02-18T11:47:39.808951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:47.822731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555491+0000","last_change":"2024-02-18T11:47:34.543371+0000","last_active":"2024-02-18T11:47:39.555491+0000","last_peered":"2024-02-18T11:47:39.555491+0000","last_clean":"2024-02-18T11:47:39.555491+0000","last_became_active":"2024-02-18T11:47:34.542194+0000","last_became_peered":"2024-02-18T11:47:34.542194+0000","last_unstale":"2024-02-18T11:47:39.555491+0000","last_undegraded":"2024-02-18T11:47:39.555491+0000","last_fullsized":"2024-02-18T11:47:39.555491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845296+0000","last_change":"2024-02-18T11:47:33.517942+0000","last_active":"2024-02-18T11:47:39.845296+0000","last_peered":"2024-02-18T11:47:39.845296+0000","last_clean":"2024-02-18T11:47:39.845296+0000","last_became_active":"2024-02-18T11:47:33.517664+0000","last_became_peered":"2024-02-18T11:47:33.517664+0000","last_unstale":"2024-02-18T11:47:39.845296+0000","last_undegraded":"2024-02-18T11:47:39.845296+0000","last_fullsized":"2024-02-18T11:47:39.845296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808749+0000","last_change":"2024-02-18T11:47:38.573311+0000","last_active":"2024-02-18T11:47:39.808749+0000","last_peered":"2024-02-18T11:47:39.808749+0000","last_clean":"2024-02-18T11:47:39.808749+0000","last_became_active":"2024-02-18T11:47:38.572888+0000","last_became_peered":"2024-02-18T11:47:38.572888+0000","last_unstale":"2024-02-18T11:47:39.808749+0000","last_undegraded":"2024-02-18T11:47:39.808749+0000","last_fullsized":"2024-02-18T11:47:39.808749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:51.214710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555516+0000","last_change":"2024-02-18T11:47:34.537515+0000","last_active":"2024-02-18T11:47:39.555516+0000","last_peered":"2024-02-18T11:47:39.555516+0000","last_clean":"2024-02-18T11:47:39.555516+0000","last_became_active":"2024-02-18T11:47:34.537235+0000","last_became_peered":"2024-02-18T11:47:34.537235+0000","last_unstale":"2024-02-18T11:47:39.555516+0000","last_undegraded":"2024-02-18T11:47:39.555516+0000","last_fullsized":"2024-02-18T11:47:39.555516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844871+0000","last_change":"2024-02-18T11:47:33.522206+0000","last_active":"2024-02-18T11:47:39.844871+0000","last_peered":"2024-02-18T11:47:39.844871+0000","last_clean":"2024-02-18T11:47:39.844871+0000","last_became_active":"2024-02-18T11:47:33.521878+0000","last_became_peered":"2024-02-18T11:47:33.521878+0000","last_unstale":"2024-02-18T11:47:39.844871+0000","last_undegraded":"2024-02-18T11:47:39.844871+0000","last_fullsized":"2024-02-18T11:47:39.844871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845354+0000","last_change":"2024-02-18T11:47:38.584826+0000","last_active":"2024-02-18T11:47:39.845354+0000","last_peered":"2024-02-18T11:47:39.845354+0000","last_clean":"2024-02-18T11:47:39.845354+0000","last_became_active":"2024-02-18T11:47:38.584161+0000","last_became_peered":"2024-02-18T11:47:38.584161+0000","last_unstale":"2024-02-18T11:47:39.845354+0000","last_undegraded":"2024-02-18T11:47:39.845354+0000","last_fullsized":"2024-02-18T11:47:39.845354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808871+0000","last_change":"2024-02-18T11:47:34.533256+0000","last_active":"2024-02-18T11:47:39.808871+0000","last_peered":"2024-02-18T11:47:39.808871+0000","last_clean":"2024-02-18T11:47:39.808871+0000","last_became_active":"2024-02-18T11:47:34.531682+0000","last_became_peered":"2024-02-18T11:47:34.531682+0000","last_unstale":"2024-02-18T11:47:39.808871+0000","last_undegraded":"2024-02-18T11:47:39.808871+0000","last_fullsized":"2024-02-18T11:47:39.808871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:09.297073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308926+0000","last_change":"2024-02-18T11:47:33.523406+0000","last_active":"2024-02-18T11:47:40.308926+0000","last_peered":"2024-02-18T11:47:40.308926+0000","last_clean":"2024-02-18T11:47:40.308926+0000","last_became_active":"2024-02-18T11:47:33.522929+0000","last_became_peered":"2024-02-18T11:47:33.522929+0000","last_unstale":"2024-02-18T11:47:40.308926+0000","last_undegraded":"2024-02-18T11:47:40.308926+0000","last_fullsized":"2024-02-18T11:47:40.308926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809106+0000","last_change":"2024-02-18T11:47:38.572268+0000","last_active":"2024-02-18T11:47:39.809106+0000","last_peered":"2024-02-18T11:47:39.809106+0000","last_clean":"2024-02-18T11:47:39.809106+0000","last_became_active":"2024-02-18T11:47:38.571698+0000","last_became_peered":"2024-02-18T11:47:38.571698+0000","last_unstale":"2024-02-18T11:47:39.809106+0000","last_undegraded":"2024-02-18T11:47:39.809106+0000","last_fullsized":"2024-02-18T11:47:39.809106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:59.447791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624439+0000","last_change":"2024-02-18T11:47:34.537052+0000","last_active":"2024-02-18T11:47:39.624439+0000","last_peered":"2024-02-18T11:47:39.624439+0000","last_clean":"2024-02-18T11:47:39.624439+0000","last_became_active":"2024-02-18T11:47:34.536782+0000","last_became_peered":"2024-02-18T11:47:34.536782+0000","last_unstale":"2024-02-18T11:47:39.624439+0000","last_undegraded":"2024-02-18T11:47:39.624439+0000","last_fullsized":"2024-02-18T11:47:39.624439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556919+0000","last_change":"2024-02-18T11:47:33.519918+0000","last_active":"2024-02-18T11:47:39.556919+0000","last_peered":"2024-02-18T11:47:39.556919+0000","last_clean":"2024-02-18T11:47:39.556919+0000","last_became_active":"2024-02-18T11:47:33.519651+0000","last_became_peered":"2024-02-18T11:47:33.519651+0000","last_unstale":"2024-02-18T11:47:39.556919+0000","last_undegraded":"2024-02-18T11:47:39.556919+0000","last_fullsized":"2024-02-18T11:47:39.556919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808561+0000","last_change":"2024-02-18T11:47:38.584734+0000","last_active":"2024-02-18T11:47:39.808561+0000","last_peered":"2024-02-18T11:47:39.808561+0000","last_clean":"2024-02-18T11:47:39.808561+0000","last_became_active":"2024-02-18T11:47:38.584299+0000","last_became_peered":"2024-02-18T11:47:38.584299+0000","last_unstale":"2024-02-18T11:47:39.808561+0000","last_undegraded":"2024-02-18T11:47:39.808561+0000","last_fullsized":"2024-02-18T11:47:39.808561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:59.168505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555556+0000","last_change":"2024-02-18T11:47:34.550205+0000","last_active":"2024-02-18T11:47:39.555556+0000","last_peered":"2024-02-18T11:47:39.555556+0000","last_clean":"2024-02-18T11:47:39.555556+0000","last_became_active":"2024-02-18T11:47:34.549966+0000","last_became_peered":"2024-02-18T11:47:34.549966+0000","last_unstale":"2024-02-18T11:47:39.555556+0000","last_undegraded":"2024-02-18T11:47:39.555556+0000","last_fullsized":"2024-02-18T11:47:39.555556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845365+0000","last_change":"2024-02-18T11:47:33.513489+0000","last_active":"2024-02-18T11:47:39.845365+0000","last_peered":"2024-02-18T11:47:39.845365+0000","last_clean":"2024-02-18T11:47:39.845365+0000","last_became_active":"2024-02-18T11:47:33.513181+0000","last_became_peered":"2024-02-18T11:47:33.513181+0000","last_unstale":"2024-02-18T11:47:39.845365+0000","last_undegraded":"2024-02-18T11:47:39.845365+0000","last_fullsized":"2024-02-18T11:47:39.845365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554976+0000","last_change":"2024-02-18T11:47:38.582725+0000","last_active":"2024-02-18T11:47:39.554976+0000","last_peered":"2024-02-18T11:47:39.554976+0000","last_clean":"2024-02-18T11:47:39.554976+0000","last_became_active":"2024-02-18T11:47:38.582354+0000","last_became_peered":"2024-02-18T11:47:38.582354+0000","last_unstale":"2024-02-18T11:47:39.554976+0000","last_undegraded":"2024-02-18T11:47:39.554976+0000","last_fullsized":"2024-02-18T11:47:39.554976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845190+0000","last_change":"2024-02-18T11:47:34.543899+0000","last_active":"2024-02-18T11:47:39.845190+0000","last_peered":"2024-02-18T11:47:39.845190+0000","last_clean":"2024-02-18T11:47:39.845190+0000","last_became_active":"2024-02-18T11:47:34.543627+0000","last_became_peered":"2024-02-18T11:47:34.543627+0000","last_unstale":"2024-02-18T11:47:39.845190+0000","last_undegraded":"2024-02-18T11:47:39.845190+0000","last_fullsized":"2024-02-18T11:47:39.845190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309676+0000","last_change":"2024-02-18T11:47:33.523004+0000","last_active":"2024-02-18T11:47:40.309676+0000","last_peered":"2024-02-18T11:47:40.309676+0000","last_clean":"2024-02-18T11:47:40.309676+0000","last_became_active":"2024-02-18T11:47:33.522170+0000","last_became_peered":"2024-02-18T11:47:33.522170+0000","last_unstale":"2024-02-18T11:47:40.309676+0000","last_undegraded":"2024-02-18T11:47:40.309676+0000","last_fullsized":"2024-02-18T11:47:40.309676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:12.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624145+0000","last_change":"2024-02-18T11:47:38.582313+0000","last_active":"2024-02-18T11:47:39.624145+0000","last_peered":"2024-02-18T11:47:39.624145+0000","last_clean":"2024-02-18T11:47:39.624145+0000","last_became_active":"2024-02-18T11:47:38.581439+0000","last_became_peered":"2024-02-18T11:47:38.581439+0000","last_unstale":"2024-02-18T11:47:39.624145+0000","last_undegraded":"2024-02-18T11:47:39.624145+0000","last_fullsized":"2024-02-18T11:47:39.624145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172507+0000","last_change":"2024-02-18T11:47:34.546405+0000","last_active":"2024-02-18T11:47:40.172507+0000","last_peered":"2024-02-18T11:47:40.172507+0000","last_clean":"2024-02-18T11:47:40.172507+0000","last_became_active":"2024-02-18T11:47:34.545110+0000","last_became_peered":"2024-02-18T11:47:34.545110+0000","last_unstale":"2024-02-18T11:47:40.172507+0000","last_undegraded":"2024-02-18T11:47:40.172507+0000","last_fullsized":"2024-02-18T11:47:40.172507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845298+0000","last_change":"2024-02-18T11:47:33.515238+0000","last_active":"2024-02-18T11:47:39.845298+0000","last_peered":"2024-02-18T11:47:39.845298+0000","last_clean":"2024-02-18T11:47:39.845298+0000","last_became_active":"2024-02-18T11:47:33.514792+0000","last_became_peered":"2024-02-18T11:47:33.514792+0000","last_unstale":"2024-02-18T11:47:39.845298+0000","last_undegraded":"2024-02-18T11:47:39.845298+0000","last_fullsized":"2024-02-18T11:47:39.845298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845086+0000","last_change":"2024-02-18T11:47:38.569586+0000","last_active":"2024-02-18T11:47:39.845086+0000","last_peered":"2024-02-18T11:47:39.845086+0000","last_clean":"2024-02-18T11:47:39.845086+0000","last_became_active":"2024-02-18T11:47:38.568487+0000","last_became_peered":"2024-02-18T11:47:38.568487+0000","last_unstale":"2024-02-18T11:47:39.845086+0000","last_undegraded":"2024-02-18T11:47:39.845086+0000","last_fullsized":"2024-02-18T11:47:39.845086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809233+0000","last_change":"2024-02-18T11:47:34.533351+0000","last_active":"2024-02-18T11:47:39.809233+0000","last_peered":"2024-02-18T11:47:39.809233+0000","last_clean":"2024-02-18T11:47:39.809233+0000","last_became_active":"2024-02-18T11:47:34.532667+0000","last_became_peered":"2024-02-18T11:47:34.532667+0000","last_unstale":"2024-02-18T11:47:39.809233+0000","last_undegraded":"2024-02-18T11:47:39.809233+0000","last_fullsized":"2024-02-18T11:47:39.809233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:26.230805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172223+0000","last_change":"2024-02-18T11:47:33.526859+0000","last_active":"2024-02-18T11:47:40.172223+0000","last_peered":"2024-02-18T11:47:40.172223+0000","last_clean":"2024-02-18T11:47:40.172223+0000","last_became_active":"2024-02-18T11:47:33.525839+0000","last_became_peered":"2024-02-18T11:47:33.525839+0000","last_unstale":"2024-02-18T11:47:40.172223+0000","last_undegraded":"2024-02-18T11:47:40.172223+0000","last_fullsized":"2024-02-18T11:47:40.172223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845416+0000","last_change":"2024-02-18T11:47:38.562544+0000","last_active":"2024-02-18T11:47:39.845416+0000","last_peered":"2024-02-18T11:47:39.845416+0000","last_clean":"2024-02-18T11:47:39.845416+0000","last_became_active":"2024-02-18T11:47:38.562171+0000","last_became_peered":"2024-02-18T11:47:38.562171+0000","last_unstale":"2024-02-18T11:47:39.845416+0000","last_undegraded":"2024-02-18T11:47:39.845416+0000","last_fullsized":"2024-02-18T11:47:39.845416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844931+0000","last_change":"2024-02-18T11:47:34.544098+0000","last_active":"2024-02-18T11:47:39.844931+0000","last_peered":"2024-02-18T11:47:39.844931+0000","last_clean":"2024-02-18T11:47:39.844931+0000","last_became_active":"2024-02-18T11:47:34.543279+0000","last_became_peered":"2024-02-18T11:47:34.543279+0000","last_unstale":"2024-02-18T11:47:39.844931+0000","last_undegraded":"2024-02-18T11:47:39.844931+0000","last_fullsized":"2024-02-18T11:47:39.844931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554630+0000","last_change":"2024-02-18T11:47:33.518923+0000","last_active":"2024-02-18T11:47:39.554630+0000","last_peered":"2024-02-18T11:47:39.554630+0000","last_clean":"2024-02-18T11:47:39.554630+0000","last_became_active":"2024-02-18T11:47:33.518591+0000","last_became_peered":"2024-02-18T11:47:33.518591+0000","last_unstale":"2024-02-18T11:47:39.554630+0000","last_undegraded":"2024-02-18T11:47:39.554630+0000","last_fullsized":"2024-02-18T11:47:39.554630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845451+0000","last_change":"2024-02-18T11:47:38.571577+0000","last_active":"2024-02-18T11:47:39.845451+0000","last_peered":"2024-02-18T11:47:39.845451+0000","last_clean":"2024-02-18T11:47:39.845451+0000","last_became_active":"2024-02-18T11:47:38.571165+0000","last_became_peered":"2024-02-18T11:47:38.571165+0000","last_unstale":"2024-02-18T11:47:39.845451+0000","last_undegraded":"2024-02-18T11:47:39.845451+0000","last_fullsized":"2024-02-18T11:47:39.845451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.623861+0000","last_change":"2024-02-18T11:47:34.533365+0000","last_active":"2024-02-18T11:47:39.623861+0000","last_peered":"2024-02-18T11:47:39.623861+0000","last_clean":"2024-02-18T11:47:39.623861+0000","last_became_active":"2024-02-18T11:47:34.533103+0000","last_became_peered":"2024-02-18T11:47:34.533103+0000","last_unstale":"2024-02-18T11:47:39.623861+0000","last_undegraded":"2024-02-18T11:47:39.623861+0000","last_fullsized":"2024-02-18T11:47:39.623861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:13.022192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308945+0000","last_change":"2024-02-18T11:47:33.523599+0000","last_active":"2024-02-18T11:47:40.308945+0000","last_peered":"2024-02-18T11:47:40.308945+0000","last_clean":"2024-02-18T11:47:40.308945+0000","last_became_active":"2024-02-18T11:47:33.523327+0000","last_became_peered":"2024-02-18T11:47:33.523327+0000","last_unstale":"2024-02-18T11:47:40.308945+0000","last_undegraded":"2024-02-18T11:47:40.308945+0000","last_fullsized":"2024-02-18T11:47:40.308945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624414+0000","last_change":"2024-02-18T11:47:38.581803+0000","last_active":"2024-02-18T11:47:39.624414+0000","last_peered":"2024-02-18T11:47:39.624414+0000","last_clean":"2024-02-18T11:47:39.624414+0000","last_became_active":"2024-02-18T11:47:38.581401+0000","last_became_peered":"2024-02-18T11:47:38.581401+0000","last_unstale":"2024-02-18T11:47:39.624414+0000","last_undegraded":"2024-02-18T11:47:39.624414+0000","last_fullsized":"2024-02-18T11:47:39.624414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556062+0000","last_change":"2024-02-18T11:47:34.544673+0000","last_active":"2024-02-18T11:47:39.556062+0000","last_peered":"2024-02-18T11:47:39.556062+0000","last_clean":"2024-02-18T11:47:39.556062+0000","last_became_active":"2024-02-18T11:47:34.544427+0000","last_became_peered":"2024-02-18T11:47:34.544427+0000","last_unstale":"2024-02-18T11:47:39.556062+0000","last_undegraded":"2024-02-18T11:47:39.556062+0000","last_fullsized":"2024-02-18T11:47:39.556062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172045+0000","last_change":"2024-02-18T11:47:33.522514+0000","last_active":"2024-02-18T11:47:40.172045+0000","last_peered":"2024-02-18T11:47:40.172045+0000","last_clean":"2024-02-18T11:47:40.172045+0000","last_became_active":"2024-02-18T11:47:33.522205+0000","last_became_peered":"2024-02-18T11:47:33.522205+0000","last_unstale":"2024-02-18T11:47:40.172045+0000","last_undegraded":"2024-02-18T11:47:40.172045+0000","last_fullsized":"2024-02-18T11:47:40.172045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309116+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:47:40.309116+0000","last_peered":"2024-02-18T11:47:40.309116+0000","last_clean":"2024-02-18T11:47:40.309116+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:47:40.309116+0000","last_undegraded":"2024-02-18T11:47:40.309116+0000","last_fullsized":"2024-02-18T11:47:40.309116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556628+0000","last_change":"2024-02-18T11:47:38.584484+0000","last_active":"2024-02-18T11:47:39.556628+0000","last_peered":"2024-02-18T11:47:39.556628+0000","last_clean":"2024-02-18T11:47:39.556628+0000","last_became_active":"2024-02-18T11:47:38.584089+0000","last_became_peered":"2024-02-18T11:47:38.584089+0000","last_unstale":"2024-02-18T11:47:39.556628+0000","last_undegraded":"2024-02-18T11:47:39.556628+0000","last_fullsized":"2024-02-18T11:47:39.556628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309440+0000","last_change":"2024-02-18T11:47:34.548586+0000","last_active":"2024-02-18T11:47:40.309440+0000","last_peered":"2024-02-18T11:47:40.309440+0000","last_clean":"2024-02-18T11:47:40.309440+0000","last_became_active":"2024-02-18T11:47:34.548315+0000","last_became_peered":"2024-02-18T11:47:34.548315+0000","last_unstale":"2024-02-18T11:47:40.309440+0000","last_undegraded":"2024-02-18T11:47:40.309440+0000","last_fullsized":"2024-02-18T11:47:40.309440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554890+0000","last_change":"2024-02-18T11:47:33.526959+0000","last_active":"2024-02-18T11:47:39.554890+0000","last_peered":"2024-02-18T11:47:39.554890+0000","last_clean":"2024-02-18T11:47:39.554890+0000","last_became_active":"2024-02-18T11:47:33.526632+0000","last_became_peered":"2024-02-18T11:47:33.526632+0000","last_unstale":"2024-02-18T11:47:39.554890+0000","last_undegraded":"2024-02-18T11:47:39.554890+0000","last_fullsized":"2024-02-18T11:47:39.554890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556483+0000","last_change":"2024-02-18T11:47:38.581910+0000","last_active":"2024-02-18T11:47:39.556483+0000","last_peered":"2024-02-18T11:47:39.556483+0000","last_clean":"2024-02-18T11:47:39.556483+0000","last_became_active":"2024-02-18T11:47:38.581463+0000","last_became_peered":"2024-02-18T11:47:38.581463+0000","last_unstale":"2024-02-18T11:47:39.556483+0000","last_undegraded":"2024-02-18T11:47:39.556483+0000","last_fullsized":"2024-02-18T11:47:39.556483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172330+0000","last_change":"2024-02-18T11:47:34.545863+0000","last_active":"2024-02-18T11:47:40.172330+0000","last_peered":"2024-02-18T11:47:40.172330+0000","last_clean":"2024-02-18T11:47:40.172330+0000","last_became_active":"2024-02-18T11:47:34.545142+0000","last_became_peered":"2024-02-18T11:47:34.545142+0000","last_unstale":"2024-02-18T11:47:40.172330+0000","last_undegraded":"2024-02-18T11:47:40.172330+0000","last_fullsized":"2024-02-18T11:47:40.172330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845452+0000","last_change":"2024-02-18T11:47:33.519784+0000","last_active":"2024-02-18T11:47:39.845452+0000","last_peered":"2024-02-18T11:47:39.845452+0000","last_clean":"2024-02-18T11:47:39.845452+0000","last_became_active":"2024-02-18T11:47:33.519495+0000","last_became_peered":"2024-02-18T11:47:33.519495+0000","last_unstale":"2024-02-18T11:47:39.845452+0000","last_undegraded":"2024-02-18T11:47:39.845452+0000","last_fullsized":"2024-02-18T11:47:39.845452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624233+0000","last_change":"2024-02-18T11:47:38.575385+0000","last_active":"2024-02-18T11:47:39.624233+0000","last_peered":"2024-02-18T11:47:39.624233+0000","last_clean":"2024-02-18T11:47:39.624233+0000","last_became_active":"2024-02-18T11:47:38.574974+0000","last_became_peered":"2024-02-18T11:47:38.574974+0000","last_unstale":"2024-02-18T11:47:39.624233+0000","last_undegraded":"2024-02-18T11:47:39.624233+0000","last_fullsized":"2024-02-18T11:47:39.624233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309750+0000","last_change":"2024-02-18T11:47:34.550657+0000","last_active":"2024-02-18T11:47:40.309750+0000","last_peered":"2024-02-18T11:47:40.309750+0000","last_clean":"2024-02-18T11:47:40.309750+0000","last_became_active":"2024-02-18T11:47:34.550286+0000","last_became_peered":"2024-02-18T11:47:34.550286+0000","last_unstale":"2024-02-18T11:47:40.309750+0000","last_undegraded":"2024-02-18T11:47:40.309750+0000","last_fullsized":"2024-02-18T11:47:40.309750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555138+0000","last_change":"2024-02-18T11:47:33.518941+0000","last_active":"2024-02-18T11:47:39.555138+0000","last_peered":"2024-02-18T11:47:39.555138+0000","last_clean":"2024-02-18T11:47:39.555138+0000","last_became_active":"2024-02-18T11:47:33.518644+0000","last_became_peered":"2024-02-18T11:47:33.518644+0000","last_unstale":"2024-02-18T11:47:39.555138+0000","last_undegraded":"2024-02-18T11:47:39.555138+0000","last_fullsized":"2024-02-18T11:47:39.555138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171756+0000","last_change":"2024-02-18T11:47:38.582832+0000","last_active":"2024-02-18T11:47:40.171756+0000","last_peered":"2024-02-18T11:47:40.171756+0000","last_clean":"2024-02-18T11:47:40.171756+0000","last_became_active":"2024-02-18T11:47:38.582436+0000","last_became_peered":"2024-02-18T11:47:38.582436+0000","last_unstale":"2024-02-18T11:47:40.171756+0000","last_undegraded":"2024-02-18T11:47:40.171756+0000","last_fullsized":"2024-02-18T11:47:40.171756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309643+0000","last_change":"2024-02-18T11:47:34.544607+0000","last_active":"2024-02-18T11:47:40.309643+0000","last_peered":"2024-02-18T11:47:40.309643+0000","last_clean":"2024-02-18T11:47:40.309643+0000","last_became_active":"2024-02-18T11:47:34.544257+0000","last_became_peered":"2024-02-18T11:47:34.544257+0000","last_unstale":"2024-02-18T11:47:40.309643+0000","last_undegraded":"2024-02-18T11:47:40.309643+0000","last_fullsized":"2024-02-18T11:47:40.309643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809144+0000","last_change":"2024-02-18T11:47:33.516233+0000","last_active":"2024-02-18T11:47:39.809144+0000","last_peered":"2024-02-18T11:47:39.809144+0000","last_clean":"2024-02-18T11:47:39.809144+0000","last_became_active":"2024-02-18T11:47:33.515830+0000","last_became_peered":"2024-02-18T11:47:33.515830+0000","last_unstale":"2024-02-18T11:47:39.809144+0000","last_undegraded":"2024-02-18T11:47:39.809144+0000","last_fullsized":"2024-02-18T11:47:39.809144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:43:01.501097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808439+0000","last_change":"2024-02-18T11:47:38.573381+0000","last_active":"2024-02-18T11:47:39.808439+0000","last_peered":"2024-02-18T11:47:39.808439+0000","last_clean":"2024-02-18T11:47:39.808439+0000","last_became_active":"2024-02-18T11:47:38.572824+0000","last_became_peered":"2024-02-18T11:47:38.572824+0000","last_unstale":"2024-02-18T11:47:39.808439+0000","last_undegraded":"2024-02-18T11:47:39.808439+0000","last_fullsized":"2024-02-18T11:47:39.808439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:11.073343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309074+0000","last_change":"2024-02-18T11:47:34.543813+0000","last_active":"2024-02-18T11:47:40.309074+0000","last_peered":"2024-02-18T11:47:40.309074+0000","last_clean":"2024-02-18T11:47:40.309074+0000","last_became_active":"2024-02-18T11:47:34.543520+0000","last_became_peered":"2024-02-18T11:47:34.543520+0000","last_unstale":"2024-02-18T11:47:40.309074+0000","last_undegraded":"2024-02-18T11:47:40.309074+0000","last_fullsized":"2024-02-18T11:47:40.309074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554409+0000","last_change":"2024-02-18T11:47:33.519311+0000","last_active":"2024-02-18T11:47:39.554409+0000","last_peered":"2024-02-18T11:47:39.554409+0000","last_clean":"2024-02-18T11:47:39.554409+0000","last_became_active":"2024-02-18T11:47:33.518936+0000","last_became_peered":"2024-02-18T11:47:33.518936+0000","last_unstale":"2024-02-18T11:47:39.554409+0000","last_undegraded":"2024-02-18T11:47:39.554409+0000","last_fullsized":"2024-02-18T11:47:39.554409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624339+0000","last_change":"2024-02-18T11:47:38.568770+0000","last_active":"2024-02-18T11:47:39.624339+0000","last_peered":"2024-02-18T11:47:39.624339+0000","last_clean":"2024-02-18T11:47:39.624339+0000","last_became_active":"2024-02-18T11:47:38.568394+0000","last_became_peered":"2024-02-18T11:47:38.568394+0000","last_unstale":"2024-02-18T11:47:39.624339+0000","last_undegraded":"2024-02-18T11:47:39.624339+0000","last_fullsized":"2024-02-18T11:47:39.624339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:04.112912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556127+0000","last_change":"2024-02-18T11:47:34.530166+0000","last_active":"2024-02-18T11:47:39.556127+0000","last_peered":"2024-02-18T11:47:39.556127+0000","last_clean":"2024-02-18T11:47:39.556127+0000","last_became_active":"2024-02-18T11:47:34.529875+0000","last_became_peered":"2024-02-18T11:47:34.529875+0000","last_unstale":"2024-02-18T11:47:39.556127+0000","last_undegraded":"2024-02-18T11:47:39.556127+0000","last_fullsized":"2024-02-18T11:47:39.556127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556190+0000","last_change":"2024-02-18T11:47:33.519858+0000","last_active":"2024-02-18T11:47:39.556190+0000","last_peered":"2024-02-18T11:47:39.556190+0000","last_clean":"2024-02-18T11:47:39.556190+0000","last_became_active":"2024-02-18T11:47:33.519239+0000","last_became_peered":"2024-02-18T11:47:33.519239+0000","last_unstale":"2024-02-18T11:47:39.556190+0000","last_undegraded":"2024-02-18T11:47:39.556190+0000","last_fullsized":"2024-02-18T11:47:39.556190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555696+0000","last_change":"2024-02-18T11:47:38.583646+0000","last_active":"2024-02-18T11:47:39.555696+0000","last_peered":"2024-02-18T11:47:39.555696+0000","last_clean":"2024-02-18T11:47:39.555696+0000","last_became_active":"2024-02-18T11:47:38.583283+0000","last_became_peered":"2024-02-18T11:47:38.583283+0000","last_unstale":"2024-02-18T11:47:39.555696+0000","last_undegraded":"2024-02-18T11:47:39.555696+0000","last_fullsized":"2024-02-18T11:47:39.555696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309237+0000","last_change":"2024-02-18T11:47:34.549395+0000","last_active":"2024-02-18T11:47:40.309237+0000","last_peered":"2024-02-18T11:47:40.309237+0000","last_clean":"2024-02-18T11:47:40.309237+0000","last_became_active":"2024-02-18T11:47:34.549157+0000","last_became_peered":"2024-02-18T11:47:34.549157+0000","last_unstale":"2024-02-18T11:47:40.309237+0000","last_undegraded":"2024-02-18T11:47:40.309237+0000","last_fullsized":"2024-02-18T11:47:40.309237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172210+0000","last_change":"2024-02-18T11:47:33.526266+0000","last_active":"2024-02-18T11:47:40.172210+0000","last_peered":"2024-02-18T11:47:40.172210+0000","last_clean":"2024-02-18T11:47:40.172210+0000","last_became_active":"2024-02-18T11:47:33.525885+0000","last_became_peered":"2024-02-18T11:47:33.525885+0000","last_unstale":"2024-02-18T11:47:40.172210+0000","last_undegraded":"2024-02-18T11:47:40.172210+0000","last_fullsized":"2024-02-18T11:47:40.172210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555736+0000","last_change":"2024-02-18T11:47:38.581784+0000","last_active":"2024-02-18T11:47:39.555736+0000","last_peered":"2024-02-18T11:47:39.555736+0000","last_clean":"2024-02-18T11:47:39.555736+0000","last_became_active":"2024-02-18T11:47:38.581391+0000","last_became_peered":"2024-02-18T11:47:38.581391+0000","last_unstale":"2024-02-18T11:47:39.555736+0000","last_undegraded":"2024-02-18T11:47:39.555736+0000","last_fullsized":"2024-02-18T11:47:39.555736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624104+0000","last_change":"2024-02-18T11:47:34.537306+0000","last_active":"2024-02-18T11:47:39.624104+0000","last_peered":"2024-02-18T11:47:39.624104+0000","last_clean":"2024-02-18T11:47:39.624104+0000","last_became_active":"2024-02-18T11:47:34.537048+0000","last_became_peered":"2024-02-18T11:47:34.537048+0000","last_unstale":"2024-02-18T11:47:39.624104+0000","last_undegraded":"2024-02-18T11:47:39.624104+0000","last_fullsized":"2024-02-18T11:47:39.624104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:19:55.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624290+0000","last_change":"2024-02-18T11:47:33.518377+0000","last_active":"2024-02-18T11:47:39.624290+0000","last_peered":"2024-02-18T11:47:39.624290+0000","last_clean":"2024-02-18T11:47:39.624290+0000","last_became_active":"2024-02-18T11:47:33.518105+0000","last_became_peered":"2024-02-18T11:47:33.518105+0000","last_unstale":"2024-02-18T11:47:39.624290+0000","last_undegraded":"2024-02-18T11:47:39.624290+0000","last_fullsized":"2024-02-18T11:47:39.624290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:12:42.834778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172424+0000","last_change":"2024-02-18T11:47:38.585105+0000","last_active":"2024-02-18T11:47:40.172424+0000","last_peered":"2024-02-18T11:47:40.172424+0000","last_clean":"2024-02-18T11:47:40.172424+0000","last_became_active":"2024-02-18T11:47:38.584660+0000","last_became_peered":"2024-02-18T11:47:38.584660+0000","last_unstale":"2024-02-18T11:47:40.172424+0000","last_undegraded":"2024-02-18T11:47:40.172424+0000","last_fullsized":"2024-02-18T11:47:40.172424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309312+0000","last_change":"2024-02-18T11:47:34.546219+0000","last_active":"2024-02-18T11:47:40.309312+0000","last_peered":"2024-02-18T11:47:40.309312+0000","last_clean":"2024-02-18T11:47:40.309312+0000","last_became_active":"2024-02-18T11:47:34.545961+0000","last_became_peered":"2024-02-18T11:47:34.545961+0000","last_unstale":"2024-02-18T11:47:40.309312+0000","last_undegraded":"2024-02-18T11:47:40.309312+0000","last_fullsized":"2024-02-18T11:47:40.309312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:43.783562+0000","last_change":"2024-02-18T11:47:33.511834+0000","last_active":"2024-02-18T11:47:43.783562+0000","last_peered":"2024-02-18T11:47:43.783562+0000","last_clean":"2024-02-18T11:47:43.783562+0000","last_became_active":"2024-02-18T11:47:33.511537+0000","last_became_peered":"2024-02-18T11:47:33.511537+0000","last_unstale":"2024-02-18T11:47:43.783562+0000","last_undegraded":"2024-02-18T11:47:43.783562+0000","last_fullsized":"2024-02-18T11:47:43.783562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845016+0000","last_change":"2024-02-18T11:47:38.570461+0000","last_active":"2024-02-18T11:47:39.845016+0000","last_peered":"2024-02-18T11:47:39.845016+0000","last_clean":"2024-02-18T11:47:39.845016+0000","last_became_active":"2024-02-18T11:47:38.568615+0000","last_became_peered":"2024-02-18T11:47:38.568615+0000","last_unstale":"2024-02-18T11:47:39.845016+0000","last_undegraded":"2024-02-18T11:47:39.845016+0000","last_fullsized":"2024-02-18T11:47:39.845016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808848+0000","last_change":"2024-02-18T11:47:34.541278+0000","last_active":"2024-02-18T11:47:39.808848+0000","last_peered":"2024-02-18T11:47:39.808848+0000","last_clean":"2024-02-18T11:47:39.808848+0000","last_became_active":"2024-02-18T11:47:34.540891+0000","last_became_peered":"2024-02-18T11:47:34.540891+0000","last_unstale":"2024-02-18T11:47:39.808848+0000","last_undegraded":"2024-02-18T11:47:39.808848+0000","last_fullsized":"2024-02-18T11:47:39.808848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845415+0000","last_change":"2024-02-18T11:47:33.526021+0000","last_active":"2024-02-18T11:47:39.845415+0000","last_peered":"2024-02-18T11:47:39.845415+0000","last_clean":"2024-02-18T11:47:39.845415+0000","last_became_active":"2024-02-18T11:47:33.525810+0000","last_became_peered":"2024-02-18T11:47:33.525810+0000","last_unstale":"2024-02-18T11:47:39.845415+0000","last_undegraded":"2024-02-18T11:47:39.845415+0000","last_fullsized":"2024-02-18T11:47:39.845415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809063+0000","last_change":"2024-02-18T11:47:38.568064+0000","last_active":"2024-02-18T11:47:39.809063+0000","last_peered":"2024-02-18T11:47:39.809063+0000","last_clean":"2024-02-18T11:47:39.809063+0000","last_became_active":"2024-02-18T11:47:38.567648+0000","last_became_peered":"2024-02-18T11:47:38.567648+0000","last_unstale":"2024-02-18T11:47:39.809063+0000","last_undegraded":"2024-02-18T11:47:39.809063+0000","last_fullsized":"2024-02-18T11:47:39.809063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:41.335475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624622+0000","last_change":"2024-02-18T11:47:34.546545+0000","last_active":"2024-02-18T11:47:39.624622+0000","last_peered":"2024-02-18T11:47:39.624622+0000","last_clean":"2024-02-18T11:47:39.624622+0000","last_became_active":"2024-02-18T11:47:34.545918+0000","last_became_peered":"2024-02-18T11:47:34.545918+0000","last_unstale":"2024-02-18T11:47:39.624622+0000","last_undegraded":"2024-02-18T11:47:39.624622+0000","last_fullsized":"2024-02-18T11:47:39.624622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172369+0000","last_change":"2024-02-18T11:47:33.526119+0000","last_active":"2024-02-18T11:47:40.172369+0000","last_peered":"2024-02-18T11:47:40.172369+0000","last_clean":"2024-02-18T11:47:40.172369+0000","last_became_active":"2024-02-18T11:47:33.525722+0000","last_became_peered":"2024-02-18T11:47:33.525722+0000","last_unstale":"2024-02-18T11:47:40.172369+0000","last_undegraded":"2024-02-18T11:47:40.172369+0000","last_fullsized":"2024-02-18T11:47:40.172369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555321+0000","last_change":"2024-02-18T11:47:38.569853+0000","last_active":"2024-02-18T11:47:39.555321+0000","last_peered":"2024-02-18T11:47:39.555321+0000","last_clean":"2024-02-18T11:47:39.555321+0000","last_became_active":"2024-02-18T11:47:38.569315+0000","last_became_peered":"2024-02-18T11:47:38.569315+0000","last_unstale":"2024-02-18T11:47:39.555321+0000","last_undegraded":"2024-02-18T11:47:39.555321+0000","last_fullsized":"2024-02-18T11:47:39.555321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:22:24.799952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172285+0000","last_change":"2024-02-18T11:47:34.538399+0000","last_active":"2024-02-18T11:47:40.172285+0000","last_peered":"2024-02-18T11:47:40.172285+0000","last_clean":"2024-02-18T11:47:40.172285+0000","last_became_active":"2024-02-18T11:47:34.538133+0000","last_became_peered":"2024-02-18T11:47:34.538133+0000","last_unstale":"2024-02-18T11:47:40.172285+0000","last_undegraded":"2024-02-18T11:47:40.172285+0000","last_fullsized":"2024-02-18T11:47:40.172285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556798+0000","last_change":"2024-02-18T11:47:33.526252+0000","last_active":"2024-02-18T11:47:39.556798+0000","last_peered":"2024-02-18T11:47:39.556798+0000","last_clean":"2024-02-18T11:47:39.556798+0000","last_became_active":"2024-02-18T11:47:33.525952+0000","last_became_peered":"2024-02-18T11:47:33.525952+0000","last_unstale":"2024-02-18T11:47:39.556798+0000","last_undegraded":"2024-02-18T11:47:39.556798+0000","last_fullsized":"2024-02-18T11:47:39.556798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555129+0000","last_change":"2024-02-18T11:47:38.583952+0000","last_active":"2024-02-18T11:47:39.555129+0000","last_peered":"2024-02-18T11:47:39.555129+0000","last_clean":"2024-02-18T11:47:39.555129+0000","last_became_active":"2024-02-18T11:47:38.583579+0000","last_became_peered":"2024-02-18T11:47:38.583579+0000","last_unstale":"2024-02-18T11:47:39.555129+0000","last_undegraded":"2024-02-18T11:47:39.555129+0000","last_fullsized":"2024-02-18T11:47:39.555129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:33:29.418521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624283+0000","last_change":"2024-02-18T11:47:38.583643+0000","last_active":"2024-02-18T11:47:39.624283+0000","last_peered":"2024-02-18T11:47:39.624283+0000","last_clean":"2024-02-18T11:47:39.624283+0000","last_became_active":"2024-02-18T11:47:38.583245+0000","last_became_peered":"2024-02-18T11:47:38.583245+0000","last_unstale":"2024-02-18T11:47:39.624283+0000","last_undegraded":"2024-02-18T11:47:39.624283+0000","last_fullsized":"2024-02-18T11:47:39.624283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844764+0000","last_change":"2024-02-18T11:47:34.540661+0000","last_active":"2024-02-18T11:47:39.844764+0000","last_peered":"2024-02-18T11:47:39.844764+0000","last_clean":"2024-02-18T11:47:39.844764+0000","last_became_active":"2024-02-18T11:47:34.540342+0000","last_became_peered":"2024-02-18T11:47:34.540342+0000","last_unstale":"2024-02-18T11:47:39.844764+0000","last_undegraded":"2024-02-18T11:47:39.844764+0000","last_fullsized":"2024-02-18T11:47:39.844764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171940+0000","last_change":"2024-02-18T11:47:33.526132+0000","last_active":"2024-02-18T11:47:40.171940+0000","last_peered":"2024-02-18T11:47:40.171940+0000","last_clean":"2024-02-18T11:47:40.171940+0000","last_became_active":"2024-02-18T11:47:33.525798+0000","last_became_peered":"2024-02-18T11:47:33.525798+0000","last_unstale":"2024-02-18T11:47:40.171940+0000","last_undegraded":"2024-02-18T11:47:40.171940+0000","last_fullsized":"2024-02-18T11:47:40.171940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556839+0000","last_change":"2024-02-18T11:47:38.572273+0000","last_active":"2024-02-18T11:47:39.556839+0000","last_peered":"2024-02-18T11:47:39.556839+0000","last_clean":"2024-02-18T11:47:39.556839+0000","last_became_active":"2024-02-18T11:47:38.571852+0000","last_became_peered":"2024-02-18T11:47:38.571852+0000","last_unstale":"2024-02-18T11:47:39.556839+0000","last_undegraded":"2024-02-18T11:47:39.556839+0000","last_fullsized":"2024-02-18T11:47:39.556839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555386+0000","last_change":"2024-02-18T11:47:34.546984+0000","last_active":"2024-02-18T11:47:39.555386+0000","last_peered":"2024-02-18T11:47:39.555386+0000","last_clean":"2024-02-18T11:47:39.555386+0000","last_became_active":"2024-02-18T11:47:34.546738+0000","last_became_peered":"2024-02-18T11:47:34.546738+0000","last_unstale":"2024-02-18T11:47:39.555386+0000","last_undegraded":"2024-02-18T11:47:39.555386+0000","last_fullsized":"2024-02-18T11:47:39.555386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555662+0000","last_change":"2024-02-18T11:47:33.514025+0000","last_active":"2024-02-18T11:47:39.555662+0000","last_peered":"2024-02-18T11:47:39.555662+0000","last_clean":"2024-02-18T11:47:39.555662+0000","last_became_active":"2024-02-18T11:47:33.513762+0000","last_became_peered":"2024-02-18T11:47:33.513762+0000","last_unstale":"2024-02-18T11:47:39.555662+0000","last_undegraded":"2024-02-18T11:47:39.555662+0000","last_fullsized":"2024-02-18T11:47:39.555662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556529+0000","last_change":"2024-02-18T11:47:33.521856+0000","last_active":"2024-02-18T11:47:39.556529+0000","last_peered":"2024-02-18T11:47:39.556529+0000","last_clean":"2024-02-18T11:47:39.556529+0000","last_became_active":"2024-02-18T11:47:33.521563+0000","last_became_peered":"2024-02-18T11:47:33.521563+0000","last_unstale":"2024-02-18T11:47:39.556529+0000","last_undegraded":"2024-02-18T11:47:39.556529+0000","last_fullsized":"2024-02-18T11:47:39.556529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624276+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.624276+0000","last_peered":"2024-02-18T11:47:39.624276+0000","last_clean":"2024-02-18T11:47:39.624276+0000","last_became_active":"2024-02-18T11:47:34.536344+0000","last_became_peered":"2024-02-18T11:47:34.536344+0000","last_unstale":"2024-02-18T11:47:39.624276+0000","last_undegraded":"2024-02-18T11:47:39.624276+0000","last_fullsized":"2024-02-18T11:47:39.624276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554790+0000","last_change":"2024-02-18T11:47:38.572780+0000","last_active":"2024-02-18T11:47:39.554790+0000","last_peered":"2024-02-18T11:47:39.554790+0000","last_clean":"2024-02-18T11:47:39.554790+0000","last_became_active":"2024-02-18T11:47:38.572401+0000","last_became_peered":"2024-02-18T11:47:38.572401+0000","last_unstale":"2024-02-18T11:47:39.554790+0000","last_undegraded":"2024-02-18T11:47:39.554790+0000","last_fullsized":"2024-02-18T11:47:39.554790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:06.469768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555607+0000","last_change":"2024-02-18T11:47:33.521346+0000","last_active":"2024-02-18T11:47:39.555607+0000","last_peered":"2024-02-18T11:47:39.555607+0000","last_clean":"2024-02-18T11:47:39.555607+0000","last_became_active":"2024-02-18T11:47:33.521077+0000","last_became_peered":"2024-02-18T11:47:33.521077+0000","last_unstale":"2024-02-18T11:47:39.555607+0000","last_undegraded":"2024-02-18T11:47:39.555607+0000","last_fullsized":"2024-02-18T11:47:39.555607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808655+0000","last_change":"2024-02-18T11:47:34.535139+0000","last_active":"2024-02-18T11:47:39.808655+0000","last_peered":"2024-02-18T11:47:39.808655+0000","last_clean":"2024-02-18T11:47:39.808655+0000","last_became_active":"2024-02-18T11:47:34.534871+0000","last_became_peered":"2024-02-18T11:47:34.534871+0000","last_unstale":"2024-02-18T11:47:39.808655+0000","last_undegraded":"2024-02-18T11:47:39.808655+0000","last_fullsized":"2024-02-18T11:47:39.808655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:58.805082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556009+0000","last_change":"2024-02-18T11:47:38.566494+0000","last_active":"2024-02-18T11:47:39.556009+0000","last_peered":"2024-02-18T11:47:39.556009+0000","last_clean":"2024-02-18T11:47:39.556009+0000","last_became_active":"2024-02-18T11:47:38.565672+0000","last_became_peered":"2024-02-18T11:47:38.565672+0000","last_unstale":"2024-02-18T11:47:39.556009+0000","last_undegraded":"2024-02-18T11:47:39.556009+0000","last_fullsized":"2024-02-18T11:47:39.556009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:38:29.952609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556029+0000","last_change":"2024-02-18T11:47:34.531011+0000","last_active":"2024-02-18T11:47:39.556029+0000","last_peered":"2024-02-18T11:47:39.556029+0000","last_clean":"2024-02-18T11:47:39.556029+0000","last_became_active":"2024-02-18T11:47:34.530758+0000","last_became_peered":"2024-02-18T11:47:34.530758+0000","last_unstale":"2024-02-18T11:47:39.556029+0000","last_undegraded":"2024-02-18T11:47:39.556029+0000","last_fullsized":"2024-02-18T11:47:39.556029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554518+0000","last_change":"2024-02-18T11:47:34.543755+0000","last_active":"2024-02-18T11:47:39.554518+0000","last_peered":"2024-02-18T11:47:39.554518+0000","last_clean":"2024-02-18T11:47:39.554518+0000","last_became_active":"2024-02-18T11:47:34.543500+0000","last_became_peered":"2024-02-18T11:47:34.543500+0000","last_unstale":"2024-02-18T11:47:39.554518+0000","last_undegraded":"2024-02-18T11:47:39.554518+0000","last_fullsized":"2024-02-18T11:47:39.554518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172079+0000","last_change":"2024-02-18T11:47:34.545316+0000","last_active":"2024-02-18T11:47:40.172079+0000","last_peered":"2024-02-18T11:47:40.172079+0000","last_clean":"2024-02-18T11:47:40.172079+0000","last_became_active":"2024-02-18T11:47:34.545018+0000","last_became_peered":"2024-02-18T11:47:34.545018+0000","last_unstale":"2024-02-18T11:47:40.172079+0000","last_undegraded":"2024-02-18T11:47:40.172079+0000","last_fullsized":"2024-02-18T11:47:40.172079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624183+0000","last_change":"2024-02-18T11:47:34.546010+0000","last_active":"2024-02-18T11:47:39.624183+0000","last_peered":"2024-02-18T11:47:39.624183+0000","last_clean":"2024-02-18T11:47:39.624183+0000","last_became_active":"2024-02-18T11:47:34.545247+0000","last_became_peered":"2024-02-18T11:47:34.545247+0000","last_unstale":"2024-02-18T11:47:39.624183+0000","last_undegraded":"2024-02-18T11:47:39.624183+0000","last_fullsized":"2024-02-18T11:47:39.624183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555993+0000","last_change":"2024-02-18T11:47:34.538436+0000","last_active":"2024-02-18T11:47:39.555993+0000","last_peered":"2024-02-18T11:47:39.555993+0000","last_clean":"2024-02-18T11:47:39.555993+0000","last_became_active":"2024-02-18T11:47:34.538207+0000","last_became_peered":"2024-02-18T11:47:34.538207+0000","last_unstale":"2024-02-18T11:47:39.555993+0000","last_undegraded":"2024-02-18T11:47:39.555993+0000","last_fullsized":"2024-02-18T11:47:39.555993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845187+0000","last_change":"2024-02-18T11:47:34.542741+0000","last_active":"2024-02-18T11:47:39.845187+0000","last_peered":"2024-02-18T11:47:39.845187+0000","last_clean":"2024-02-18T11:47:39.845187+0000","last_became_active":"2024-02-18T11:47:34.542467+0000","last_became_peered":"2024-02-18T11:47:34.542467+0000","last_unstale":"2024-02-18T11:47:39.845187+0000","last_undegraded":"2024-02-18T11:47:39.845187+0000","last_fullsized":"2024-02-18T11:47:39.845187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557105+0000","last_change":"2024-02-18T11:47:34.539970+0000","last_active":"2024-02-18T11:47:39.557105+0000","last_peered":"2024-02-18T11:47:39.557105+0000","last_clean":"2024-02-18T11:47:39.557105+0000","last_became_active":"2024-02-18T11:47:34.539471+0000","last_became_peered":"2024-02-18T11:47:34.539471+0000","last_unstale":"2024-02-18T11:47:39.557105+0000","last_undegraded":"2024-02-18T11:47:39.557105+0000","last_fullsized":"2024-02-18T11:47:39.557105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555437+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.555437+0000","last_peered":"2024-02-18T11:47:39.555437+0000","last_clean":"2024-02-18T11:47:39.555437+0000","last_became_active":"2024-02-18T11:47:34.536356+0000","last_became_peered":"2024-02-18T11:47:34.536356+0000","last_unstale":"2024-02-18T11:47:39.555437+0000","last_undegraded":"2024-02-18T11:47:39.555437+0000","last_fullsized":"2024-02-18T11:47:39.555437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171840+0000","last_change":"2024-02-18T11:47:34.542717+0000","last_active":"2024-02-18T11:47:40.171840+0000","last_peered":"2024-02-18T11:47:40.171840+0000","last_clean":"2024-02-18T11:47:40.171840+0000","last_became_active":"2024-02-18T11:47:34.542470+0000","last_became_peered":"2024-02-18T11:47:34.542470+0000","last_unstale":"2024-02-18T11:47:40.171840+0000","last_undegraded":"2024-02-18T11:47:40.171840+0000","last_fullsized":"2024-02-18T11:47:40.171840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555193+0000","last_change":"2024-02-18T11:47:34.537727+0000","last_active":"2024-02-18T11:47:39.555193+0000","last_peered":"2024-02-18T11:47:39.555193+0000","last_clean":"2024-02-18T11:47:39.555193+0000","last_became_active":"2024-02-18T11:47:34.536902+0000","last_became_peered":"2024-02-18T11:47:34.536902+0000","last_unstale":"2024-02-18T11:47:39.555193+0000","last_undegraded":"2024-02-18T11:47:39.555193+0000","last_fullsized":"2024-02-18T11:47:39.555193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172026+0000","last_change":"2024-02-18T11:47:34.549313+0000","last_active":"2024-02-18T11:47:40.172026+0000","last_peered":"2024-02-18T11:47:40.172026+0000","last_clean":"2024-02-18T11:47:40.172026+0000","last_became_active":"2024-02-18T11:47:34.549010+0000","last_became_peered":"2024-02-18T11:47:34.549010+0000","last_unstale":"2024-02-18T11:47:40.172026+0000","last_undegraded":"2024-02-18T11:47:40.172026+0000","last_fullsized":"2024-02-18T11:47:40.172026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624061+0000","last_change":"2024-02-18T11:47:34.546238+0000","last_active":"2024-02-18T11:47:39.624061+0000","last_peered":"2024-02-18T11:47:39.624061+0000","last_clean":"2024-02-18T11:47:39.624061+0000","last_became_active":"2024-02-18T11:47:34.545634+0000","last_became_peered":"2024-02-18T11:47:34.545634+0000","last_unstale":"2024-02-18T11:47:39.624061+0000","last_undegraded":"2024-02-18T11:47:39.624061+0000","last_fullsized":"2024-02-18T11:47:39.624061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:18:21.067312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556482+0000","last_change":"2024-02-18T11:47:34.534576+0000","last_active":"2024-02-18T11:47:39.556482+0000","last_peered":"2024-02-18T11:47:39.556482+0000","last_clean":"2024-02-18T11:47:39.556482+0000","last_became_active":"2024-02-18T11:47:34.534232+0000","last_became_peered":"2024-02-18T11:47:34.534232+0000","last_unstale":"2024-02-18T11:47:39.556482+0000","last_undegraded":"2024-02-18T11:47:39.556482+0000","last_fullsized":"2024-02-18T11:47:39.556482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309588+0000","last_change":"2024-02-18T11:47:34.546750+0000","last_active":"2024-02-18T11:47:40.309588+0000","last_peered":"2024-02-18T11:47:40.309588+0000","last_clean":"2024-02-18T11:47:40.309588+0000","last_became_active":"2024-02-18T11:47:34.546507+0000","last_became_peered":"2024-02-18T11:47:34.546507+0000","last_unstale":"2024-02-18T11:47:40.309588+0000","last_undegraded":"2024-02-18T11:47:40.309588+0000","last_fullsized":"2024-02-18T11:47:40.309588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555043+0000","last_change":"2024-02-18T11:47:34.542028+0000","last_active":"2024-02-18T11:47:39.555043+0000","last_peered":"2024-02-18T11:47:39.555043+0000","last_clean":"2024-02-18T11:47:39.555043+0000","last_became_active":"2024-02-18T11:47:34.541653+0000","last_became_peered":"2024-02-18T11:47:34.541653+0000","last_unstale":"2024-02-18T11:47:39.555043+0000","last_undegraded":"2024-02-18T11:47:39.555043+0000","last_fullsized":"2024-02-18T11:47:39.555043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:38.652128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172499+0000","last_change":"2024-02-18T11:47:34.549989+0000","last_active":"2024-02-18T11:47:40.172499+0000","last_peered":"2024-02-18T11:47:40.172499+0000","last_clean":"2024-02-18T11:47:40.172499+0000","last_became_active":"2024-02-18T11:47:34.549661+0000","last_became_peered":"2024-02-18T11:47:34.549661+0000","last_unstale":"2024-02-18T11:47:40.172499+0000","last_undegraded":"2024-02-18T11:47:40.172499+0000","last_fullsized":"2024-02-18T11:47:40.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171732+0000","last_change":"2024-02-18T11:47:34.547081+0000","last_active":"2024-02-18T11:47:40.171732+0000","last_peered":"2024-02-18T11:47:40.171732+0000","last_clean":"2024-02-18T11:47:40.171732+0000","last_became_active":"2024-02-18T11:47:34.546812+0000","last_became_peered":"2024-02-18T11:47:34.546812+0000","last_unstale":"2024-02-18T11:47:40.171732+0000","last_undegraded":"2024-02-18T11:47:40.171732+0000","last_fullsized":"2024-02-18T11:47:40.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844848+0000","last_change":"2024-02-18T11:47:34.534907+0000","last_active":"2024-02-18T11:47:39.844848+0000","last_peered":"2024-02-18T11:47:39.844848+0000","last_clean":"2024-02-18T11:47:39.844848+0000","last_became_active":"2024-02-18T11:47:34.534661+0000","last_became_peered":"2024-02-18T11:47:34.534661+0000","last_unstale":"2024-02-18T11:47:39.844848+0000","last_undegraded":"2024-02-18T11:47:39.844848+0000","last_fullsized":"2024-02-18T11:47:39.844848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624050+0000","last_change":"2024-02-18T11:47:34.537888+0000","last_active":"2024-02-18T11:47:39.624050+0000","last_peered":"2024-02-18T11:47:39.624050+0000","last_clean":"2024-02-18T11:47:39.624050+0000","last_became_active":"2024-02-18T11:47:34.537647+0000","last_became_peered":"2024-02-18T11:47:34.537647+0000","last_unstale":"2024-02-18T11:47:39.624050+0000","last_undegraded":"2024-02-18T11:47:39.624050+0000","last_fullsized":"2024-02-18T11:47:39.624050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:02.426785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555401+0000","last_change":"2024-02-18T11:47:34.541254+0000","last_active":"2024-02-18T11:47:39.555401+0000","last_peered":"2024-02-18T11:47:39.555401+0000","last_clean":"2024-02-18T11:47:39.555401+0000","last_became_active":"2024-02-18T11:47:34.540955+0000","last_became_peered":"2024-02-18T11:47:34.540955+0000","last_unstale":"2024-02-18T11:47:39.555401+0000","last_undegraded":"2024-02-18T11:47:39.555401+0000","last_fullsized":"2024-02-18T11:47:39.555401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808837+0000","last_change":"2024-02-18T11:47:34.531217+0000","last_active":"2024-02-18T11:47:39.808837+0000","last_peered":"2024-02-18T11:47:39.808837+0000","last_clean":"2024-02-18T11:47:39.808837+0000","last_became_active":"2024-02-18T11:47:34.530120+0000","last_became_peered":"2024-02-18T11:47:34.530120+0000","last_unstale":"2024-02-18T11:47:39.808837+0000","last_undegraded":"2024-02-18T11:47:39.808837+0000","last_fullsized":"2024-02-18T11:47:39.808837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:43.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556645+0000","last_change":"2024-02-18T11:47:34.544075+0000","last_active":"2024-02-18T11:47:39.556645+0000","last_peered":"2024-02-18T11:47:39.556645+0000","last_clean":"2024-02-18T11:47:39.556645+0000","last_became_active":"2024-02-18T11:47:34.543761+0000","last_became_peered":"2024-02-18T11:47:34.543761+0000","last_unstale":"2024-02-18T11:47:39.556645+0000","last_undegraded":"2024-02-18T11:47:39.556645+0000","last_fullsized":"2024-02-18T11:47:39.556645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557031+0000","last_change":"2024-02-18T11:47:34.546387+0000","last_active":"2024-02-18T11:47:39.557031+0000","last_peered":"2024-02-18T11:47:39.557031+0000","last_clean":"2024-02-18T11:47:39.557031+0000","last_became_active":"2024-02-18T11:47:34.546115+0000","last_became_peered":"2024-02-18T11:47:34.546115+0000","last_unstale":"2024-02-18T11:47:39.557031+0000","last_undegraded":"2024-02-18T11:47:39.557031+0000","last_fullsized":"2024-02-18T11:47:39.557031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309697+0000","last_change":"2024-02-18T11:47:34.552880+0000","last_active":"2024-02-18T11:47:40.309697+0000","last_peered":"2024-02-18T11:47:40.309697+0000","last_clean":"2024-02-18T11:47:40.309697+0000","last_became_active":"2024-02-18T11:47:34.552530+0000","last_became_peered":"2024-02-18T11:47:34.552530+0000","last_unstale":"2024-02-18T11:47:40.309697+0000","last_undegraded":"2024-02-18T11:47:40.309697+0000","last_fullsized":"2024-02-18T11:47:40.309697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555106+0000","last_change":"2024-02-18T11:47:34.545727+0000","last_active":"2024-02-18T11:47:39.555106+0000","last_peered":"2024-02-18T11:47:39.555106+0000","last_clean":"2024-02-18T11:47:39.555106+0000","last_became_active":"2024-02-18T11:47:34.545475+0000","last_became_peered":"2024-02-18T11:47:34.545475+0000","last_unstale":"2024-02-18T11:47:39.555106+0000","last_undegraded":"2024-02-18T11:47:39.555106+0000","last_fullsized":"2024-02-18T11:47:39.555106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309012+0000","last_change":"2024-02-18T11:47:34.547696+0000","last_active":"2024-02-18T11:47:40.309012+0000","last_peered":"2024-02-18T11:47:40.309012+0000","last_clean":"2024-02-18T11:47:40.309012+0000","last_became_active":"2024-02-18T11:47:34.547448+0000","last_became_peered":"2024-02-18T11:47:34.547448+0000","last_unstale":"2024-02-18T11:47:40.309012+0000","last_undegraded":"2024-02-18T11:47:40.309012+0000","last_fullsized":"2024-02-18T11:47:40.309012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171835+0000","last_change":"2024-02-18T11:47:34.548053+0000","last_active":"2024-02-18T11:47:40.171835+0000","last_peered":"2024-02-18T11:47:40.171835+0000","last_clean":"2024-02-18T11:47:40.171835+0000","last_became_active":"2024-02-18T11:47:34.547181+0000","last_became_peered":"2024-02-18T11:47:34.547181+0000","last_unstale":"2024-02-18T11:47:40.171835+0000","last_undegraded":"2024-02-18T11:47:40.171835+0000","last_fullsized":"2024-02-18T11:47:40.171835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309134+0000","last_change":"2024-02-18T11:47:34.549242+0000","last_active":"2024-02-18T11:47:40.309134+0000","last_peered":"2024-02-18T11:47:40.309134+0000","last_clean":"2024-02-18T11:47:40.309134+0000","last_became_active":"2024-02-18T11:47:34.548872+0000","last_became_peered":"2024-02-18T11:47:34.548872+0000","last_unstale":"2024-02-18T11:47:40.309134+0000","last_undegraded":"2024-02-18T11:47:40.309134+0000","last_fullsized":"2024-02-18T11:47:40.309134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172083+0000","last_change":"2024-02-18T11:47:34.546466+0000","last_active":"2024-02-18T11:47:40.172083+0000","last_peered":"2024-02-18T11:47:40.172083+0000","last_clean":"2024-02-18T11:47:40.172083+0000","last_became_active":"2024-02-18T11:47:34.546218+0000","last_became_peered":"2024-02-18T11:47:34.546218+0000","last_unstale":"2024-02-18T11:47:40.172083+0000","last_undegraded":"2024-02-18T11:47:40.172083+0000","last_fullsized":"2024-02-18T11:47:40.172083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624360+0000","last_change":"2024-02-18T11:47:34.530060+0000","last_active":"2024-02-18T11:47:39.624360+0000","last_peered":"2024-02-18T11:47:39.624360+0000","last_clean":"2024-02-18T11:47:39.624360+0000","last_became_active":"2024-02-18T11:47:34.529799+0000","last_became_peered":"2024-02-18T11:47:34.529799+0000","last_unstale":"2024-02-18T11:47:39.624360+0000","last_undegraded":"2024-02-18T11:47:39.624360+0000","last_fullsized":"2024-02-18T11:47:39.624360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845225+0000","last_change":"2024-02-18T11:47:34.547273+0000","last_active":"2024-02-18T11:47:39.845225+0000","last_peered":"2024-02-18T11:47:39.845225+0000","last_clean":"2024-02-18T11:47:39.845225+0000","last_became_active":"2024-02-18T11:47:34.546984+0000","last_became_peered":"2024-02-18T11:47:34.546984+0000","last_unstale":"2024-02-18T11:47:39.845225+0000","last_undegraded":"2024-02-18T11:47:39.845225+0000","last_fullsized":"2024-02-18T11:47:39.845225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309602+0000","last_change":"2024-02-18T11:47:34.547456+0000","last_active":"2024-02-18T11:47:40.309602+0000","last_peered":"2024-02-18T11:47:40.309602+0000","last_clean":"2024-02-18T11:47:40.309602+0000","last_became_active":"2024-02-18T11:47:34.547193+0000","last_became_peered":"2024-02-18T11:47:34.547193+0000","last_unstale":"2024-02-18T11:47:40.309602+0000","last_undegraded":"2024-02-18T11:47:40.309602+0000","last_fullsized":"2024-02-18T11:47:40.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:47:57.900 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-18T11:47:57.900 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-18T11:47:57.901 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-02-18T11:47:57.901 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-02-18T11:47:57.901 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-02-18T11:47:58.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.042+0000 7f13487eb640 1 Processor -- start 2024-02-18T11:47:58.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.042+0000 7f13487eb640 1 -- start start 2024-02-18T11:47:58.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f13487eb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13400cf900 0x7f13400d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f13487eb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 0x7f1340099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f13487eb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13400d0a80 con 0x7f134015efb0 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f13487eb640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f134015f390 con 0x7f13400cf900 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f13487eb640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134009a400 con 0x7f1340099a50 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1346587640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f134015efb0 0x7f134006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45892/0 (socket says 172.21.15.116:45892) 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1346587640 1 -- 172.21.15.116:0/2499394461 learned_addr learned my addr 172.21.15.116:0/2499394461 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019164656 0 0) 0x7f13400d0a80 con 0x7f134015efb0 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1346d88640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 0x7f1340099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f132c003520 con 0x7f134015efb0 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345d86640 1 --2- 172.21.15.116:0/2499394461 >> v2:172.21.15.143:3300/0 conn(0x7f13400cf900 0x7f13400d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3955225630 0 0) 0x7f132c003520 con 0x7f134015efb0 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 msgr2=0x7f1340099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 0x7f1340099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 >> v2:172.21.15.143:3300/0 conn(0x7f13400cf900 msgr2=0x7f13400d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 --2- 172.21.15.116:0/2499394461 >> v2:172.21.15.143:3300/0 conn(0x7f13400cf900 0x7f13400d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400cfe70 con 0x7f134015efb0 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f133c001670 con 0x7f134015efb0 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 >> v1:172.21.15.116:6789/0 conn(0x7f134015efb0 legacy=0x7f134006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 0x7f132c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c003ef0 0x7f132c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c004d00 0x7f132c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134015f390 con 0x7f132c004d00 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134009a400 con 0x7f132c003ef0 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c00b9d0 con 0x7f1340099a50 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1346d88640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c004d00 0x7f132c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.043+0000 7f1346587640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 0x7f132c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1345d86640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c003ef0 0x7f132c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1346d88640 1 -- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 msgr2=0x7f132c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.040 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1346d88640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1340099a50 0x7f132c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1346d88640 1 -- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c003ef0 msgr2=0x7f132c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1346d88640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c003ef0 0x7f132c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1346d88640 1 -- 172.21.15.116:0/2499394461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c003520 con 0x7f132c004d00 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1346d88640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c004d00 0x7f132c005180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f133800e170 tx=0x7f132c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8427fdd0aaa7b33 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133802cbf0 con 0x7f132c004d00 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133800f040 con 0x7f132c004d00 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.044+0000 7f1345585640 1 -- 172.21.15.116:0/2499394461 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338034440 con 0x7f132c004d00 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f13487eb640 1 -- 172.21.15.116:0/2499394461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c004d00 msgr2=0x7f132c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f13487eb640 1 --2- 172.21.15.116:0/2499394461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c004d00 0x7f132c005180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f133800e170 tx=0x7f132c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f1346d88640 1 -- 172.21.15.116:0/2499394461 reap_dead start 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f13487eb640 1 -- 172.21.15.116:0/2499394461 shutdown_connections 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f13487eb640 1 -- 172.21.15.116:0/2499394461 >> 172.21.15.116:0/2499394461 conn(0x7f13400d1790 msgr2=0x7f13400d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f13487eb640 1 -- 172.21.15.116:0/2499394461 shutdown_connections 2024-02-18T11:47:58.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.045+0000 7f13487eb640 1 -- 172.21.15.116:0/2499394461 wait complete. 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.046+0000 7f13487eb640 1 Processor -- start 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.046+0000 7f13487eb640 1 -- start start 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f13487eb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c004d00 0x7f13400b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f13487eb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f13400af1d0 0x7f134015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f13487eb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 0x7f13401f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f13487eb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401f7820 con 0x7f13401f6e30 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f13487eb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340158290 con 0x7f132c004d00 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f13487eb640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13400af670 con 0x7f13400af1d0 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 0x7f13401f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 0x7f13401f72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57166/0 (socket says 172.21.15.116:57166) 2024-02-18T11:47:58.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 -- 172.21.15.116:0/267819668 learned_addr learned my addr 172.21.15.116:0/267819668 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1345d86640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f13400af1d0 0x7f134015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346587640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c004d00 0x7f13400b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 -- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f13400af1d0 msgr2=0x7f134015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f13400af1d0 0x7f134015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 -- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c004d00 msgr2=0x7f13400b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c004d00 0x7f13400b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 -- 172.21.15.116:0/267819668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400af950 con 0x7f13401f6e30 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.047+0000 7f1346d88640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 0x7f13401f72e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f133800e170 tx=0x7f1338009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.048+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338034d50 con 0x7f13401f6e30 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.048+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133802de80 con 0x7f13401f6e30 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.048+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13400afc30 con 0x7f13401f6e30 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.048+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133803e680 con 0x7f13401f6e30 2024-02-18T11:47:58.046 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.048+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13400b01f0 con 0x7f13401f6e30 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.049+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1340067840 con 0x7f13401f6e30 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.049+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f133803e7e0 con 0x7f13401f6e30 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.050+0000 7f1336ffd640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f132406bca0 0x7f132406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.050+0000 7f1346587640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f132406bca0 0x7f132406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.050+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f13380acbb0 con 0x7f13401f6e30 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.050+0000 7f1346587640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f132406bca0 0x7f132406e160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f133c001eb0 tx=0x7f133c0078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.054+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1338077110 con 0x7f13401f6e30 2024-02-18T11:47:58.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.269+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1340074520 con 0x7f13401f6e30 2024-02-18T11:47:58.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.272+0000 7f1336ffd640 1 -- 172.21.15.116:0/267819668 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+172339 (secure 0 0 0) 0x7f1338039020 con 0x7f13401f6e30 2024-02-18T11:47:58.268 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f132406bca0 msgr2=0x7f132406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f132406bca0 0x7f132406e160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f133c001eb0 tx=0x7f133c0078f0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 msgr2=0x7f13401f72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 0x7f13401f72e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f133800e170 tx=0x7f1338009750 comp rx=0 tx=0).stop 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 shutdown_connections 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f132406bca0 0x7f132406e160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13401f6e30 0x7f13401f72e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f13400af1d0 0x7f134015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 --2- 172.21.15.116:0/267819668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f132c004d00 0x7f13400b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 >> 172.21.15.116:0/267819668 conn(0x7f13400d1790 msgr2=0x7f13400b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:58.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 shutdown_connections 2024-02-18T11:47:58.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.275+0000 7f13487eb640 1 -- 172.21.15.116:0/267819668 wait complete. 2024-02-18T11:47:58.279 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4274496429},{"type":"v1","addr":"172.21.15.116:6833","nonce":4274496429}]},"active_addr":"172.21.15.116:6833/4274496429","active_change":"2024-02-18T11:47:14.282145+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":2519431847}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":1187173714}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":553933067}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.116:0","nonce":2890847493}]}]} 2024-02-18T11:47:58.282 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-02-18T11:47:58.282 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-02-18T11:47:58.282 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:58.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 Processor -- start 2024-02-18T11:47:58.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 -- start start 2024-02-18T11:47:58.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0780161f50 0x7f07800b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 0x7f07800b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078015e1a0 con 0x7f078015ddc0 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07800d3750 con 0x7f0780161f50 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.390+0000 7f07859e9640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07800b96a0 con 0x7f07800b6b00 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077effd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f078015ddc0 0x7f0780161840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45894/0 (socket says 172.21.15.116:45894) 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077effd640 1 -- 172.21.15.116:0/988667197 learned_addr learned my addr 172.21.15.116:0/988667197 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077f7fe640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 0x7f07800b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764303589 0 0) 0x7f0770001110 con 0x7f078015ddc0 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f0777fff640 1 --2- 172.21.15.116:0/988667197 >> v2:172.21.15.143:3300/0 conn(0x7f0780161f50 0x7f07800b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f076c003520 con 0x7f078015ddc0 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306086627 0 0) 0x7f076c003520 con 0x7f078015ddc0 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 msgr2=0x7f07800b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 0x7f07800b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 >> v2:172.21.15.143:3300/0 conn(0x7f0780161f50 msgr2=0x7f07800b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 --2- 172.21.15.116:0/988667197 >> v2:172.21.15.143:3300/0 conn(0x7f0780161f50 0x7f07800b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07801623e0 con 0x7f078015ddc0 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f0770001800 con 0x7f078015ddc0 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 >> v1:172.21.15.116:6789/0 conn(0x7f078015ddc0 legacy=0x7f0780161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f076c003970 0x7f076c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 0x7f076c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800d3750 con 0x7f076c0047c0 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800b96a0 con 0x7f076c003970 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c00b980 con 0x7f07800b6b00 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f0777fff640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077effd640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f076c003970 0x7f076c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.391+0000 7f077f7fe640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 0x7f076c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f0777fff640 1 -- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 msgr2=0x7f076c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f0777fff640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f07800b6b00 0x7f076c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f0777fff640 1 -- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f076c003970 msgr2=0x7f076c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f0777fff640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f076c003970 0x7f076c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f0777fff640 1 -- 172.21.15.116:0/988667197 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c003520 con 0x7f076c0047c0 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f0777fff640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0768009ac0 tx=0x7f076c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cd28ca892f2b1d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.392+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076802c710 con 0x7f076c0047c0 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076802c870 con 0x7f076c0047c0 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f077e7fc640 1 -- 172.21.15.116:0/988667197 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076802cc00 con 0x7f076c0047c0 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f07859e9640 1 -- 172.21.15.116:0/988667197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f076c0047c0 msgr2=0x7f076c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f07859e9640 1 --2- 172.21.15.116:0/988667197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0768009ac0 tx=0x7f076c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f077f7fe640 1 -- 172.21.15.116:0/988667197 reap_dead start 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f07859e9640 1 -- 172.21.15.116:0/988667197 shutdown_connections 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f07859e9640 1 -- 172.21.15.116:0/988667197 >> 172.21.15.116:0/988667197 conn(0x7f07800d0870 msgr2=0x7f07800d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:58.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f07859e9640 1 -- 172.21.15.116:0/988667197 shutdown_connections 2024-02-18T11:47:58.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.393+0000 7f07859e9640 1 -- 172.21.15.116:0/988667197 wait complete. 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.394+0000 7f07859e9640 1 Processor -- start 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.394+0000 7f07859e9640 1 -- start start 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f07859e9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f076c0047c0 0x7f07800c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f07859e9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07800caaa0 0x7f07800cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f07859e9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 0x7f07800ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f07859e9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c77b0 con 0x7f07800caf00 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f07859e9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800cd100 con 0x7f07800caaa0 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f07859e9640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07800c6610 con 0x7f076c0047c0 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 0x7f07800ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 0x7f07800ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57184/0 (socket says 172.21.15.116:57184) 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 -- 172.21.15.116:0/2956638674 learned_addr learned my addr 172.21.15.116:0/2956638674 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077effd640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f076c0047c0 0x7f07800c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f0777fff640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07800caaa0 0x7f07800cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 -- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f076c0047c0 msgr2=0x7f07800c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f076c0047c0 0x7f07800c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 -- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07800caaa0 msgr2=0x7f07800cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07800caaa0 0x7f07800cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.395+0000 7f077f7fe640 1 -- 172.21.15.116:0/2956638674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780160560 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.396+0000 7f077f7fe640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 0x7f07800ccbc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07800b68d0 tx=0x7f077800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.396+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077800e040 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.396+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077800f040 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.396+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778003bd0 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.396+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0780160840 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.396+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07800c79c0 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.397+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078006c400 con 0x7f07800caf00 2024-02-18T11:47:58.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.397+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f0778003d30 con 0x7f07800caf00 2024-02-18T11:47:58.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.397+0000 7f077cff9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f076006bca0 0x7f076006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.397+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f077808b1d0 con 0x7f07800caf00 2024-02-18T11:47:58.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.398+0000 7f077effd640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f076006bca0 0x7f076006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.398+0000 7f077effd640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f076006bca0 0x7f076006e160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0770001050 tx=0x7f0770009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.402+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07780555e0 con 0x7f07800caf00 2024-02-18T11:47:58.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.577+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07800d16e0 con 0x7f07800caf00 2024-02-18T11:47:58.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.578+0000 7f077cff9640 1 -- 172.21.15.116:0/2956638674 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f0778058ca0 con 0x7f07800caf00 2024-02-18T11:47:58.571 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:58.572 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:58.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f076006bca0 msgr2=0x7f076006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f076006bca0 0x7f076006e160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0770001050 tx=0x7f0770009040 comp rx=0 tx=0).stop 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 msgr2=0x7f07800ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 0x7f07800ccbc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07800b68d0 tx=0x7f077800f580 comp rx=0 tx=0).stop 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 shutdown_connections 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f076006bca0 0x7f076006e160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07800caf00 0x7f07800ccbc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f07800caaa0 0x7f07800cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 --2- 172.21.15.116:0/2956638674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f076c0047c0 0x7f07800c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 >> 172.21.15.116:0/2956638674 conn(0x7f07800d0870 msgr2=0x7f07800b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 shutdown_connections 2024-02-18T11:47:58.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.580+0000 7f07859e9640 1 -- 172.21.15.116:0/2956638674 wait complete. 2024-02-18T11:47:58.585 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-02-18T11:47:58.585 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:47:58.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.692+0000 7fdc1ee76640 1 Processor -- start 2024-02-18T11:47:58.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.692+0000 7fdc1ee76640 1 -- start start 2024-02-18T11:47:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1ee76640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc1806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1ee76640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdc180b36f0 0x7fdc180b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1ee76640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc180d0400 con 0x7fdc18160e90 2024-02-18T11:47:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1ee76640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1806b530 con 0x7fdc180b36f0 2024-02-18T11:47:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1ee76640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180d14a0 con 0x7fdc1806ab60 2024-02-18T11:47:58.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1cc12640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdc18160e90 0x7fdc180afac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45900/0 (socket says 172.21.15.116:45900) 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1d413640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdc180b36f0 0x7fdc180b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1cc12640 1 -- 172.21.15.116:0/1905521940 learned_addr learned my addr 172.21.15.116:0/1905521940 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc17fff640 1 --2- 172.21.15.116:0/1905521940 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc1806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc177fe640 1 -- 172.21.15.116:0/1905521940 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697559689 0 0) 0x7fdc180d0400 con 0x7fdc18160e90 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc177fe640 1 -- 172.21.15.116:0/1905521940 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc00003520 con 0x7fdc18160e90 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1d413640 1 -- 172.21.15.116:0/1905521940 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 msgr2=0x7fdc1806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1d413640 1 --2- 172.21.15.116:0/1905521940 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc1806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1d413640 1 -- 172.21.15.116:0/1905521940 >> v1:172.21.15.116:6789/0 conn(0x7fdc18160e90 legacy=0x7fdc180afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.693+0000 7fdc1d413640 1 -- 172.21.15.116:0/1905521940 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180d06e0 con 0x7fdc180b36f0 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.694+0000 7fdc1d413640 1 --2- 172.21.15.116:0/1905521940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc180b36f0 0x7fdc180b3b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009d90 tx=0x7fdc1002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e643b761378217 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.694+0000 7fdc177fe640 1 -- 172.21.15.116:0/1905521940 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1002cbc0 con 0x7fdc180b36f0 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.694+0000 7fdc177fe640 1 -- 172.21.15.116:0/1905521940 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1002cd20 con 0x7fdc180b36f0 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.694+0000 7fdc177fe640 1 -- 172.21.15.116:0/1905521940 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10034660 con 0x7fdc180b36f0 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 -- 172.21.15.116:0/1905521940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc180b36f0 msgr2=0x7fdc180b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/1905521940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc180b36f0 0x7fdc180b3b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009d90 tx=0x7fdc1002be10 comp rx=0 tx=0).stop 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 -- 172.21.15.116:0/1905521940 shutdown_connections 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/1905521940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc180b36f0 0x7fdc180b3b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/1905521940 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc1806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 -- 172.21.15.116:0/1905521940 >> 172.21.15.116:0/1905521940 conn(0x7fdc180b23f0 msgr2=0x7fdc180b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 -- 172.21.15.116:0/1905521940 shutdown_connections 2024-02-18T11:47:58.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.695+0000 7fdc1ee76640 1 -- 172.21.15.116:0/1905521940 wait complete. 2024-02-18T11:47:58.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.696+0000 7fdc1ee76640 1 Processor -- start 2024-02-18T11:47:58.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.696+0000 7fdc1ee76640 1 -- start start 2024-02-18T11:47:58.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc181593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180b36f0 0x7fdc181f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 0x7fdc18161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc181589d0 con 0x7fdc18160e90 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc181f40d0 con 0x7fdc180b36f0 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc181f43b0 con 0x7fdc1806ab60 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 0x7fdc18161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 0x7fdc18161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57210/0 (socket says 172.21.15.116:57210) 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 -- 172.21.15.116:0/336271912 learned_addr learned my addr 172.21.15.116:0/336271912 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 -- 172.21.15.116:0/336271912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 msgr2=0x7fdc181593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc181593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 -- 172.21.15.116:0/336271912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180b36f0 msgr2=0x7fdc181f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180b36f0 0x7fdc181f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 -- 172.21.15.116:0/336271912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc10009a70 con 0x7fdc18160e90 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1d413640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 0x7fdc18161560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009d90 tx=0x7fdc10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10040650 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1002ce90 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.697+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc18158be0 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.698+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc100409f0 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.698+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1815a040 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.699+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc18159100 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.699+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fdc10040b50 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.699+0000 7fdc157fa640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdc0406bca0 0x7fdc0406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.700+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdc100ad3f0 con 0x7fdc18160e90 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.700+0000 7fdc1cc12640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdc0406bca0 0x7fdc0406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:58.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.700+0000 7fdc1cc12640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdc0406bca0 0x7fdc0406e160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdc08000f60 tx=0x7fdc080078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:58.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.704+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc18159100 con 0x7fdc18160e90 2024-02-18T11:47:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.874+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc18159100 con 0x7fdc18160e90 2024-02-18T11:47:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.875+0000 7fdc157fa640 1 -- 172.21.15.116:0/336271912 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fdc18159100 con 0x7fdc18160e90 2024-02-18T11:47:58.869 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:47:58.869 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.877+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdc0406bca0 msgr2=0x7fdc0406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.877+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdc0406bca0 0x7fdc0406e160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdc08000f60 tx=0x7fdc080078f0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.877+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 msgr2=0x7fdc18161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.877+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 0x7fdc18161560 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009d90 tx=0x7fdc10004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 shutdown_connections 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdc0406bca0 0x7fdc0406e160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc18160e90 0x7fdc18161560 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc180b36f0 0x7fdc181f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 --2- 172.21.15.116:0/336271912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdc1806ab60 0x7fdc181593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 >> 172.21.15.116:0/336271912 conn(0x7fdc180b23f0 msgr2=0x7fdc180b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 shutdown_connections 2024-02-18T11:47:58.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:58.878+0000 7fdc1ee76640 1 -- 172.21.15.116:0/336271912 wait complete. 2024-02-18T11:47:58.882 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-02-18T11:47:58.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-02-18T11:47:58.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-02-18T11:47:58.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-02-18T11:47:58.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-02-18T11:47:58.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-02-18T11:47:58.885 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-02-18T11:47:58.885 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-02-18T11:47:59.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.061+0000 7fb94e23b640 1 Processor -- start 2024-02-18T11:47:59.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.061+0000 7fb94e23b640 1 -- start start 2024-02-18T11:47:59.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.061+0000 7f7d676e1640 1 Processor -- start 2024-02-18T11:47:59.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb94e23b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb940009180 0x7fb940052ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb94e23b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940049060 0x7fb9400eb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb94e23b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb940009560 con 0x7fb940048c30 2024-02-18T11:47:59.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb94e23b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb940036b80 con 0x7fb940009180 2024-02-18T11:47:59.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb94e23b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb940035960 con 0x7fb940049060 2024-02-18T11:47:59.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb94d239640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb940009180 0x7fb940052ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7fb94d239640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb940009180 0x7fb940052ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53898/0 (socket says 172.21.15.116:53898) 2024-02-18T11:47:59.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f9c3849d640 1 Processor -- start 2024-02-18T11:47:59.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7fb94d239640 1 -- 172.21.15.116:0/3245375342 learned_addr learned my addr 172.21.15.116:0/3245375342 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f9c3849d640 1 -- start start 2024-02-18T11:47:59.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb685402640 1 Processor -- start 2024-02-18T11:47:59.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7fb685402640 1 -- start start 2024-02-18T11:47:59.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f814a6b5640 1 Processor -- start 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7f7d676e1640 1 -- start start 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d676e1640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b4240 0x7f7d60162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d676e1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d600b25f0 0x7f7d6006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d676e1640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d600711c0 0x7f7d60071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d676e1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d601634b0 con 0x7f7d600b25f0 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d676e1640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d60159490 con 0x7f7d600711c0 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d676e1640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60071b70 con 0x7f7d600b4240 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f7d64c7c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d600b25f0 0x7f7d6006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f7d64c7c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d600b25f0 0x7f7d6006e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57224/0 (socket says 172.21.15.116:57224) 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f7d64c7c640 1 -- 172.21.15.116:0/1670713069 learned_addr learned my addr 172.21.15.116:0/1670713069 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f7d6547d640 1 --2- 172.21.15.116:0/1670713069 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b4240 0x7f7d60162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7f6024965640 1 Processor -- start 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.062+0000 7f6024965640 1 -- start start 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f6024965640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6018050140 0x7f6018038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f6024965640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6018005f60 con 0x7f60180df030 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f6024965640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60180e38e0 con 0x7f60180df410 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f6024965640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6018006200 con 0x7f6018050140 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f601ed9d640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f60180df410 0x7f6018006a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:45290/0 (socket says 172.21.15.116:45290) 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f601ed9d640 1 -- 172.21.15.116:0/4181030552 learned_addr learned my addr 172.21.15.116:0/4181030552 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:59.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f601dd9b640 1 --2- 172.21.15.116:0/4181030552 >> v2:172.21.15.116:3300/0 conn(0x7f6018050140 0x7f6018038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7fb94d239640 1 -- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940049060 msgr2=0x7fb9400eb9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-18T11:47:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7fb94d239640 1 --2- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940049060 0x7fb9400eb9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7fb94d239640 1 -- 172.21.15.116:0/3245375342 >> v1:172.21.15.143:6789/0 conn(0x7fb940048c30 legacy=0x7fb9400eb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7fb94d239640 1 -- 172.21.15.116:0/3245375342 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb940035c20 con 0x7fb940009180 2024-02-18T11:47:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f273d8ed640 1 Processor -- start 2024-02-18T11:47:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f273d8ed640 1 -- start start 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb685402640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb68006f2b0 0x7fb6800b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb685402640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6800b4170 con 0x7fb6800718d0 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb685402640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6800b3b20 con 0x7fb680153e20 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb685402640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800b4b40 con 0x7fb68006f2b0 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb67f7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb6800718d0 0x7fb6800b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45912/0 (socket says 172.21.15.116:45912) 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb67f7fe640 1 -- 172.21.15.116:0/3985088376 learned_addr learned my addr 172.21.15.116:0/3985088376 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.063+0000 7f814a6b5640 1 -- start start 2024-02-18T11:47:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f9c3849d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c30161fa0 0x7f9c300b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f9c3849d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c3006c740 0x7f9c300d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f9c3849d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c300b9020 con 0x7f9c300cf9f0 2024-02-18T11:47:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f9c3849d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c300b8a60 con 0x7f9c3006c740 2024-02-18T11:47:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7f9c3849d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300b8d40 con 0x7f9c30161fa0 2024-02-18T11:47:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d65c7e640 1 --2- 172.21.15.116:0/1670713069 >> v2:172.21.15.143:3300/0 conn(0x7f7d600711c0 0x7f7d60071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.064+0000 7fb94d239640 1 --2- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940009180 0x7fb940052ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb944009870 tx=0x7fb94402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33b5bbaef1c9e37 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb93ffff640 1 -- 172.21.15.116:0/3245375342 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944031870 con 0x7fb940009180 2024-02-18T11:47:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb93ffff640 1 -- 172.21.15.116:0/3245375342 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9440319d0 con 0x7fb940009180 2024-02-18T11:47:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb93ffff640 1 -- 172.21.15.116:0/3245375342 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944031d60 con 0x7fb940009180 2024-02-18T11:47:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb67effd640 1 --2- 172.21.15.116:0/3985088376 >> v2:172.21.15.143:3300/0 conn(0x7fb68006f2b0 0x7fb6800b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f9c36239640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c30161fa0 0x7f9c300b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d64c7c640 1 -- 172.21.15.116:0/1670713069 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b4240 msgr2=0x7f7d60162f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d64c7c640 1 --2- 172.21.15.116:0/1670713069 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b4240 0x7f7d60162f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d64c7c640 1 -- 172.21.15.116:0/1670713069 >> v2:172.21.15.143:3300/0 conn(0x7f7d600711c0 msgr2=0x7f7d60071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d64c7c640 1 --2- 172.21.15.116:0/1670713069 >> v2:172.21.15.143:3300/0 conn(0x7f7d600711c0 0x7f7d60071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d64c7c640 1 -- 172.21.15.116:0/1670713069 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6006d220 con 0x7f7d600b25f0 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f9c36a3a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c3006c740 0x7f9c300d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f9c36a3a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c3006c740 0x7f9c300d1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57228/0 (socket says 172.21.15.116:57228) 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f9c36a3a640 1 -- 172.21.15.116:0/4200827166 learned_addr learned my addr 172.21.15.116:0/4200827166 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f814a6b5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f81440718d0 0x7f8144162f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f814a6b5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814406d220 con 0x7f81440b25f0 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f814a6b5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81440b3a00 con 0x7f81440b4240 2024-02-18T11:47:59.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f814a6b5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81441634b0 con 0x7f81440718d0 2024-02-18T11:47:59.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb67dffb640 1 -- 172.21.15.116:0/3985088376 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527714881 0 0) 0x7fb6800b3b20 con 0x7fb680153e20 2024-02-18T11:47:59.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb67dffb640 1 -- 172.21.15.116:0/3985088376 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb668003520 con 0x7fb680153e20 2024-02-18T11:47:59.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f7d65c7e640 1 --2- 172.21.15.116:0/1670713069 >> v2:172.21.15.143:3300/0 conn(0x7f7d600711c0 0x7f7d60071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:59.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f8148c52640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f81440718d0 0x7f8144162f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f8148c52640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f81440718d0 0x7f8144162f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53930/0 (socket says 172.21.15.116:53930) 2024-02-18T11:47:59.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f8148c52640 1 -- 172.21.15.116:0/3120404132 learned_addr learned my addr 172.21.15.116:0/3120404132 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f273d8ed640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f27381450d0 0x7f273806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f273d8ed640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f27380c4cc0 0x7f27380b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f8148c52640 1 -- 172.21.15.116:0/3120404132 >> v1:172.21.15.143:6789/0 conn(0x7f81440b4240 legacy=0x7f814406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f8148c52640 1 -- 172.21.15.116:0/3120404132 >> v1:172.21.15.116:6789/0 conn(0x7f81440b25f0 legacy=0x7f81440711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f8148c52640 1 -- 172.21.15.116:0/3120404132 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440b2ad0 con 0x7f81440718d0 2024-02-18T11:47:59.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f2737fff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f273806d600 0x7f27380c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:45314/0 (socket says 172.21.15.116:45314) 2024-02-18T11:47:59.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f2737fff640 1 -- 172.21.15.116:0/3840317846 learned_addr learned my addr 172.21.15.116:0/3840317846 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:59.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 -- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940009180 msgr2=0x7fb940052ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273d0ec640 1 --2- 172.21.15.116:0/3840317846 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f27380c4cc0 0x7f27380b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 --2- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940009180 0x7fb940052ef0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb944009870 tx=0x7fb94402b910 comp rx=0 tx=0).stop 2024-02-18T11:47:59.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27381454b0 con 0x7f273806d600 2024-02-18T11:47:59.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 -- 172.21.15.116:0/3245375342 shutdown_connections 2024-02-18T11:47:59.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273806dab0 con 0x7f27381450d0 2024-02-18T11:47:59.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27380c5300 con 0x7f27380c4cc0 2024-02-18T11:47:59.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f601dd9b640 1 -- 172.21.15.116:0/4181030552 >> v1:172.21.15.143:6789/0 conn(0x7f60180df410 legacy=0x7f6018006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 --2- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940049060 0x7fb9400eb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f601dd9b640 1 -- 172.21.15.116:0/4181030552 >> v1:172.21.15.116:6789/0 conn(0x7f60180df030 legacy=0x7f601804fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7f601dd9b640 1 -- 172.21.15.116:0/4181030552 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180064e0 con 0x7f6018050140 2024-02-18T11:47:59.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 --2- 172.21.15.116:0/3245375342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940009180 0x7fb940052ef0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 -- 172.21.15.116:0/3245375342 >> 172.21.15.116:0/3245375342 conn(0x7fb94001ef30 msgr2=0x7fb940055c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb67effd640 1 -- 172.21.15.116:0/3985088376 >> v1:172.21.15.143:6789/0 conn(0x7fb680153e20 legacy=0x7fb6800711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb67effd640 1 -- 172.21.15.116:0/3985088376 >> v1:172.21.15.116:6789/0 conn(0x7fb6800718d0 legacy=0x7fb6800b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.065+0000 7fb67effd640 1 -- 172.21.15.116:0/3985088376 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800b3e00 con 0x7fb68006f2b0 2024-02-18T11:47:59.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273c8eb640 1 --2- 172.21.15.116:0/3840317846 >> v2:172.21.15.116:3300/0 conn(0x7f27381450d0 0x7f273806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb67effd640 1 --2- 172.21.15.116:0/3985088376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 0x7fb6800b4600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6700097a0 tx=0x7fb67002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c2d18ef18d79c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f7d64c7c640 1 --2- 172.21.15.116:0/1670713069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600b25f0 0x7f7d6006e560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d601624f0 tx=0x7f7d5802bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9ddb6640e85731 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273c8eb640 1 -- 172.21.15.116:0/3840317846 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f27380c4cc0 msgr2=0x7f27380b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273c8eb640 1 --2- 172.21.15.116:0/3840317846 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f27380c4cc0 0x7f27380b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273c8eb640 1 -- 172.21.15.116:0/3840317846 >> v1:172.21.15.143:6789/0 conn(0x7f273806d600 legacy=0x7f27380c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f273c8eb640 1 -- 172.21.15.116:0/3840317846 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273815f4a0 con 0x7f27381450d0 2024-02-18T11:47:59.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 -- 172.21.15.116:0/3245375342 shutdown_connections 2024-02-18T11:47:59.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7fb94e23b640 1 -- 172.21.15.116:0/3245375342 wait complete. 2024-02-18T11:47:59.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7fb67dffb640 1 -- 172.21.15.116:0/3985088376 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670030840 con 0x7fb68006f2b0 2024-02-18T11:47:59.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f9c36a3a640 1 -- 172.21.15.116:0/4200827166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c30161fa0 msgr2=0x7f9c300b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f9c36a3a640 1 --2- 172.21.15.116:0/4200827166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c30161fa0 0x7f9c300b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f9c36a3a640 1 -- 172.21.15.116:0/4200827166 >> v1:172.21.15.143:6789/0 conn(0x7f9c300cf9f0 legacy=0x7f9c3006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.066+0000 7f9c36a3a640 1 -- 172.21.15.116:0/4200827166 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300d1690 con 0x7f9c3006c740 2024-02-18T11:47:59.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f8148c52640 1 --2- 172.21.15.116:0/3120404132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440718d0 0x7f8144162f70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e50 tx=0x7f813c02be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56439d39892a87a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f8142ffd640 1 -- 172.21.15.116:0/3120404132 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c02c9f0 con 0x7f81440718d0 2024-02-18T11:47:59.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7fb67dffb640 1 -- 172.21.15.116:0/3985088376 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6700309a0 con 0x7fb68006f2b0 2024-02-18T11:47:59.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f699d3b5640 1 Processor -- start 2024-02-18T11:47:59.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f699d3b5640 1 -- start start 2024-02-18T11:47:59.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f8142ffd640 1 -- 172.21.15.116:0/3120404132 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c02cb50 con 0x7f81440718d0 2024-02-18T11:47:59.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f8142ffd640 1 -- 172.21.15.116:0/3120404132 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c02c9f0 con 0x7f81440718d0 2024-02-18T11:47:59.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f7d57fff640 1 -- 172.21.15.116:0/1670713069 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802caa0 con 0x7f7d600b25f0 2024-02-18T11:47:59.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f7d57fff640 1 -- 172.21.15.116:0/1670713069 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5802cc00 con 0x7f7d600b25f0 2024-02-18T11:47:59.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f7d57fff640 1 -- 172.21.15.116:0/1670713069 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580343f0 con 0x7f7d600b25f0 2024-02-18T11:47:59.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f601dd9b640 1 --2- 172.21.15.116:0/4181030552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6018050140 0x7f6018038170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f602004fcd0 tx=0x7f60200a7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e4b4dddaafb07 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7f601d59a640 1 -- 172.21.15.116:0/4181030552 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200b2070 con 0x7f6018050140 2024-02-18T11:47:59.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f601d59a640 1 -- 172.21.15.116:0/4181030552 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60200ad620 con 0x7f6018050140 2024-02-18T11:47:59.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.067+0000 7fb67dffb640 1 -- 172.21.15.116:0/3985088376 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670030d70 con 0x7fb68006f2b0 2024-02-18T11:47:59.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f9c36a3a640 1 --2- 172.21.15.116:0/4200827166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c3006c740 0x7f9c300d1e90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009870 tx=0x7f9c2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2ea726a3060a62 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 -- 172.21.15.116:0/3985088376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 msgr2=0x7fb6800b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb94e23b640 1 Processor -- start 2024-02-18T11:47:59.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 --2- 172.21.15.116:0/3985088376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 0x7fb6800b4600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb6700097a0 tx=0x7fb67002b980 comp rx=0 tx=0).stop 2024-02-18T11:47:59.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 -- 172.21.15.116:0/3985088376 shutdown_connections 2024-02-18T11:47:59.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f814a6b5640 1 -- 172.21.15.116:0/3120404132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440718d0 msgr2=0x7f8144162f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f814a6b5640 1 --2- 172.21.15.116:0/3120404132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440718d0 0x7f8144162f70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e50 tx=0x7f813c02be50 comp rx=0 tx=0).stop 2024-02-18T11:47:59.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb94e23b640 1 -- start start 2024-02-18T11:47:59.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f9c35237640 1 -- 172.21.15.116:0/4200827166 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24031640 con 0x7f9c3006c740 2024-02-18T11:47:59.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f814a6b5640 1 -- 172.21.15.116:0/3120404132 shutdown_connections 2024-02-18T11:47:59.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f814a6b5640 1 --2- 172.21.15.116:0/3120404132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440718d0 0x7f8144162f70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f814a6b5640 1 -- 172.21.15.116:0/3120404132 >> 172.21.15.116:0/3120404132 conn(0x7f8144067000 msgr2=0x7f814406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f9c35237640 1 -- 172.21.15.116:0/4200827166 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c240317a0 con 0x7f9c3006c740 2024-02-18T11:47:59.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f9c35237640 1 -- 172.21.15.116:0/4200827166 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24031b30 con 0x7f9c3006c740 2024-02-18T11:47:59.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 --2- 172.21.15.116:0/3985088376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 0x7fb6800b4600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 -- 172.21.15.116:0/3985088376 >> 172.21.15.116:0/3985088376 conn(0x7fb680067000 msgr2=0x7fb68006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f7d676e1640 1 -- 172.21.15.116:0/1670713069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600b25f0 msgr2=0x7f7d6006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f7d676e1640 1 --2- 172.21.15.116:0/1670713069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600b25f0 0x7f7d6006e560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7d601624f0 tx=0x7f7d5802bf00 comp rx=0 tx=0).stop 2024-02-18T11:47:59.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f699d3b5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69980b93b0 0x7f69980ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f601d59a640 1 -- 172.21.15.116:0/4181030552 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60200b2070 con 0x7f6018050140 2024-02-18T11:47:59.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f6024965640 1 -- 172.21.15.116:0/4181030552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6018050140 msgr2=0x7f6018038170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f6024965640 1 --2- 172.21.15.116:0/4181030552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6018050140 0x7f6018038170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f602004fcd0 tx=0x7f60200a7800 comp rx=0 tx=0).stop 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f7d676e1640 1 -- 172.21.15.116:0/1670713069 shutdown_connections 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f7d676e1640 1 --2- 172.21.15.116:0/1670713069 >> v2:172.21.15.143:3300/0 conn(0x7f7d600711c0 0x7f7d60071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f7d676e1640 1 --2- 172.21.15.116:0/1670713069 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600b25f0 0x7f7d6006e560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f7d676e1640 1 --2- 172.21.15.116:0/1670713069 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b4240 0x7f7d60162f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f7d676e1640 1 -- 172.21.15.116:0/1670713069 >> 172.21.15.116:0/1670713069 conn(0x7f7d60067000 msgr2=0x7f7d6006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7f273c8eb640 1 --2- 172.21.15.116:0/3840317846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27381450d0 0x7f273806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27280097a0 tx=0x7f272802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b6a0c90713a7c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6996ffd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f699815dba0 0x7f6998161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45946/0 (socket says 172.21.15.116:45946) 2024-02-18T11:47:59.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6024965640 1 -- 172.21.15.116:0/4181030552 shutdown_connections 2024-02-18T11:47:59.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6024965640 1 --2- 172.21.15.116:0/4181030552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6018050140 0x7f6018038170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6024965640 1 -- 172.21.15.116:0/4181030552 >> 172.21.15.116:0/4181030552 conn(0x7f6018009fe0 msgr2=0x7f601800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6996ffd640 1 -- 172.21.15.116:0/2998597958 learned_addr learned my addr 172.21.15.116:0/2998597958 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:59.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f27377fe640 1 -- 172.21.15.116:0/3840317846 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272802d040 con 0x7f27381450d0 2024-02-18T11:47:59.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f27377fe640 1 -- 172.21.15.116:0/3840317846 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2728031810 con 0x7f27381450d0 2024-02-18T11:47:59.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f27377fe640 1 -- 172.21.15.116:0/3840317846 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728031ba0 con 0x7f27381450d0 2024-02-18T11:47:59.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6024965640 1 -- 172.21.15.116:0/4181030552 shutdown_connections 2024-02-18T11:47:59.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f69967fc640 1 --2- 172.21.15.116:0/2998597958 >> v2:172.21.15.116:3300/0 conn(0x7f69980b93b0 0x7f69980ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980d21b0 con 0x7f699815dba0 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f6024965640 1 -- 172.21.15.116:0/4181030552 wait complete. 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94e23b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940009180 0x7fb9400e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94e23b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940048c30 0x7fb94017dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94e23b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb940049060 0x7fb94004d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94e23b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400518e0 con 0x7fb940049060 2024-02-18T11:47:59.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94e23b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400e92a0 con 0x7fb940048c30 2024-02-18T11:47:59.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94e23b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94017e2b0 con 0x7fb940009180 2024-02-18T11:47:59.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f9c3849d640 1 -- 172.21.15.116:0/4200827166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c3006c740 msgr2=0x7f9c300d1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f9c3849d640 1 --2- 172.21.15.116:0/4200827166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c3006c740 0x7f9c300d1e90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009870 tx=0x7f9c2402b8f0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69980cf140 con 0x7f69980ced30 2024-02-18T11:47:59.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69980d1e60 con 0x7f69980b93b0 2024-02-18T11:47:59.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f7d676e1640 1 -- 172.21.15.116:0/1670713069 shutdown_connections 2024-02-18T11:47:59.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f7d676e1640 1 -- 172.21.15.116:0/1670713069 wait complete. 2024-02-18T11:47:59.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94ca38640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940048c30 0x7fb94017dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94ca38640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940048c30 0x7fb94017dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53940/0 (socket says 172.21.15.116:53940) 2024-02-18T11:47:59.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94ca38640 1 -- 172.21.15.116:0/3010974406 learned_addr learned my addr 172.21.15.116:0/3010974406 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f814a6b5640 1 -- 172.21.15.116:0/3120404132 shutdown_connections 2024-02-18T11:47:59.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f814a6b5640 1 -- 172.21.15.116:0/3120404132 wait complete. 2024-02-18T11:47:59.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7fb94d239640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940009180 0x7fb9400e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.069+0000 7f9c3849d640 1 -- 172.21.15.116:0/4200827166 shutdown_connections 2024-02-18T11:47:59.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f9c3849d640 1 --2- 172.21.15.116:0/4200827166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c3006c740 0x7f9c300d1e90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f9c3849d640 1 --2- 172.21.15.116:0/4200827166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c30161fa0 0x7f9c300b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f9c3849d640 1 -- 172.21.15.116:0/4200827166 >> 172.21.15.116:0/4200827166 conn(0x7f9c300d3a20 msgr2=0x7f9c300d3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27381450d0 msgr2=0x7f273806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f273d8ed640 1 --2- 172.21.15.116:0/3840317846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27381450d0 0x7f273806d0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27280097a0 tx=0x7f272802b8b0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94ca38640 1 -- 172.21.15.116:0/3010974406 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940009180 msgr2=0x7fb9400e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94ca38640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb940009180 0x7fb9400e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94ca38640 1 -- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb940049060 msgr2=0x7fb94004d8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94da3a640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb940049060 0x7fb94004d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94ca38640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb940049060 0x7fb94004d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94ca38640 1 -- 172.21.15.116:0/3010974406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9440094d0 con 0x7fb940048c30 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94da3a640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb940049060 0x7fb94004d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb94ca38640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940048c30 0x7fb94017dd70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb94802d370 tx=0x7fb948068690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 shutdown_connections 2024-02-18T11:47:59.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f273d8ed640 1 --2- 172.21.15.116:0/3840317846 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f27380c4cc0 0x7f27380b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f273d8ed640 1 --2- 172.21.15.116:0/3840317846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27381450d0 0x7f273806d0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 >> 172.21.15.116:0/3840317846 conn(0x7f2738066f20 msgr2=0x7f273806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948073d80 con 0x7fb940048c30 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948092400 con 0x7fb940048c30 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948092810 con 0x7fb940048c30 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb94e23b640 1 -- 172.21.15.116:0/3010974406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94004d2a0 con 0x7fb940048c30 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb94e23b640 1 -- 172.21.15.116:0/3010974406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb940051290 con 0x7fb940048c30 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 -- 172.21.15.116:0/3985088376 shutdown_connections 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.068+0000 7fb685402640 1 -- 172.21.15.116:0/3985088376 wait complete. 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb685402640 1 Processor -- start 2024-02-18T11:47:59.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7fb685402640 1 -- start start 2024-02-18T11:47:59.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7f6024965640 1 Processor -- start 2024-02-18T11:47:59.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7f6024965640 1 -- start start 2024-02-18T11:47:59.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7f814a6b5640 1 Processor -- start 2024-02-18T11:47:59.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f7d676e1640 1 Processor -- start 2024-02-18T11:47:59.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7f814a6b5640 1 -- start start 2024-02-18T11:47:59.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f7d676e1640 1 -- start start 2024-02-18T11:47:59.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f69967fc640 1 -- 172.21.15.116:0/2998597958 >> v1:172.21.15.143:6789/0 conn(0x7f69980ced30 legacy=0x7f69980d2640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-18T11:47:59.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f69967fc640 1 -- 172.21.15.116:0/2998597958 >> v1:172.21.15.116:6789/0 conn(0x7f699815dba0 legacy=0x7f6998161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.070+0000 7f69967fc640 1 -- 172.21.15.116:0/2998597958 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6998162050 con 0x7f69980b93b0 2024-02-18T11:47:59.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7f69967fc640 1 --2- 172.21.15.116:0/2998597958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980b93b0 0x7f69980ce7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69880097a0 tx=0x7f698802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff425b86be0861fe server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 shutdown_connections 2024-02-18T11:47:59.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f9c3849d640 1 -- 172.21.15.116:0/4200827166 shutdown_connections 2024-02-18T11:47:59.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb94e23b640 1 -- 172.21.15.116:0/3010974406 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9400060b0 con 0x7fb940048c30 2024-02-18T11:47:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb685402640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 0x7fb6800ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb685402640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6800718d0 0x7fb68014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb685402640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb680153e20 0x7fb6800a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb685402640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800abd00 con 0x7fb6800718d0 2024-02-18T11:47:59.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb685402640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800a8550 con 0x7fb68006f2b0 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb685402640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800a8830 con 0x7fb680153e20 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb67e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6800718d0 0x7fb68014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb67e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6800718d0 0x7fb68014a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57266/0 (socket says 172.21.15.116:57266) 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.071+0000 7fb67e7fc640 1 -- 172.21.15.116:0/645835164 learned_addr learned my addr 172.21.15.116:0/645835164 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb67e7fc640 1 -- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb680153e20 msgr2=0x7fb6800a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb67e7fc640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb680153e20 0x7fb6800a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb67e7fc640 1 -- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 msgr2=0x7fb6800ab7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb67e7fc640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb68006f2b0 0x7fb6800ab7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb67e7fc640 1 -- 172.21.15.116:0/645835164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb670009400 con 0x7fb6800718d0 2024-02-18T11:47:59.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7fb67e7fc640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6800718d0 0x7fb68014a430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c800 tx=0x7fb67800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f814a6b5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81440718d0 0x7f81441f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f814a6b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81440b25f0 0x7f8144165ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f814a6b5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440b4240 0x7f814414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f814a6b5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814415eac0 con 0x7f81440b25f0 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f814a6b5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814415de40 con 0x7f81440b4240 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f814a6b5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814415e120 con 0x7f81440718d0 2024-02-18T11:47:59.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.072+0000 7f273d8ed640 1 -- 172.21.15.116:0/3840317846 wait complete. 2024-02-18T11:47:59.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f6024965640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6018050140 0x7f60180db640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f6024965640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60180df030 0x7f601804bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f81437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81440b25f0 0x7f8144165ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f6024965640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60180df410 0x7f60180e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f8148c52640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440b4240 0x7f814414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f8148c52640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440b4240 0x7f814414b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53954/0 (socket says 172.21.15.116:53954) 2024-02-18T11:47:59.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f8148c52640 1 -- 172.21.15.116:0/2109945141 learned_addr learned my addr 172.21.15.116:0/2109945141 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f601ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60180df410 0x7f60180e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678003b70 con 0x7fb6800718d0 2024-02-18T11:47:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678003cd0 con 0x7fb6800718d0 2024-02-18T11:47:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f601ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60180df410 0x7f60180e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57282/0 (socket says 172.21.15.116:57282) 2024-02-18T11:47:59.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f8148c52640 1 -- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81440718d0 msgr2=0x7f81441f7cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:47:59.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f601ed9d640 1 -- 172.21.15.116:0/1856450914 learned_addr learned my addr 172.21.15.116:0/1856450914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018049770 con 0x7f60180df410 2024-02-18T11:47:59.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180dc100 con 0x7f6018050140 2024-02-18T11:47:59.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601804e6a0 con 0x7f60180df030 2024-02-18T11:47:59.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f8148c52640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81440718d0 0x7f81441f7cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f8148c52640 1 -- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81440b25f0 msgr2=0x7f8144165ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f273d8ed640 1 Processor -- start 2024-02-18T11:47:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d676e1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600711c0 0x7f7d6013efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d676e1640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b25f0 0x7f7d601596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f8148c52640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81440b25f0 0x7f8144165ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d676e1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d600b4240 0x7f7d6015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d676e1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6015aa20 con 0x7f7d600711c0 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d676e1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d601585e0 con 0x7f7d600b4240 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d676e1640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d60159da0 con 0x7f7d600b25f0 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d65c7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d600b4240 0x7f7d6015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d64c7c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b25f0 0x7f7d601596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d65c7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d600b4240 0x7f7d6015ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53964/0 (socket says 172.21.15.116:53964) 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7f7d65c7e640 1 -- 172.21.15.116:0/3190171230 learned_addr learned my addr 172.21.15.116:0/3190171230 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f273d8ed640 1 -- start start 2024-02-18T11:47:59.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800aa70 con 0x7fb6800718d0 2024-02-18T11:47:59.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb685402640 1 -- 172.21.15.116:0/645835164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6800a8b10 con 0x7fb6800718d0 2024-02-18T11:47:59.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb685402640 1 -- 172.21.15.116:0/645835164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6800a90d0 con 0x7fb6800718d0 2024-02-18T11:47:59.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f8148c52640 1 -- 172.21.15.116:0/2109945141 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c009b30 con 0x7f81440b4240 2024-02-18T11:47:59.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d64c7c640 1 -- 172.21.15.116:0/3190171230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d600b4240 msgr2=0x7f7d6015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d64c7c640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d600b4240 0x7f7d6015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d64c7c640 1 -- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600711c0 msgr2=0x7f7d6013efb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d64c7c640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d600711c0 0x7f7d6013efb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d64c7c640 1 -- 172.21.15.116:0/3190171230 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58009b30 con 0x7f7d600b25f0 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f81437fe640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81440b25f0 0x7f8144165ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.073+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fb948092970 con 0x7fb940048c30 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb93dffb640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb92c06bd70 0x7fb92c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb948105000 con 0x7fb940048c30 2024-02-18T11:47:59.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb93dffb640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] conn(0x7fb92c0754f0 0x7fb92c077930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 --> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb92c078000 con 0x7fb92c0754f0 2024-02-18T11:47:59.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9481075c0 con 0x7fb940048c30 2024-02-18T11:47:59.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d64c7c640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b25f0 0x7f7d601596c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d58032000 tx=0x7f7d58004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5803e450 con 0x7f7d600b25f0 2024-02-18T11:47:59.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f9c3849d640 1 -- 172.21.15.116:0/4200827166 wait complete. 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7fb94d239640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb92c06bd70 0x7fb92c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f601ed9d640 1 -- 172.21.15.116:0/1856450914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60180df030 msgr2=0x7f601804bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f601ed9d640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60180df030 0x7f601804bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f601ed9d640 1 -- 172.21.15.116:0/1856450914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6018050140 msgr2=0x7f60180db640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f601ed9d640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6018050140 0x7f60180db640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f601ed9d640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601804e960 con 0x7f60180df410 2024-02-18T11:47:59.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f601ed9d640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60180df410 0x7f60180e0cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f601000ab00 tx=0x7f6010007170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f6995ffb640 1 -- 172.21.15.116:0/2998597958 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988031830 con 0x7f69980b93b0 2024-02-18T11:47:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f6995ffb640 1 -- 172.21.15.116:0/2998597958 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6988031990 con 0x7f69980b93b0 2024-02-18T11:47:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f6995ffb640 1 -- 172.21.15.116:0/2998597958 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6988031d20 con 0x7f69980b93b0 2024-02-18T11:47:59.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7fb94da3a640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] conn(0x7fb92c0754f0 0x7fb92c077930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5803e5b0 con 0x7f7d600b25f0 2024-02-18T11:47:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5803e760 con 0x7f7d600b25f0 2024-02-18T11:47:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d676e1640 1 -- 172.21.15.116:0/3190171230 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6015a080 con 0x7f7d600b25f0 2024-02-18T11:47:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.074+0000 7f7d676e1640 1 -- 172.21.15.116:0/3190171230 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6015a640 con 0x7f7d600b25f0 2024-02-18T11:47:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f273d8ed640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f273806d600 0x7f273813b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb680154520 con 0x7fb6800718d0 2024-02-18T11:47:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f273d8ed640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27380c4cc0 0x7f273815e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7fb94da3a640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] conn(0x7fb92c0754f0 0x7fb92c077930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7fb94d239640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb92c06bd70 0x7fb92c06e230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb944036000 tx=0x7fb94403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000e220 con 0x7f60180df410 2024-02-18T11:47:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6010012030 con 0x7f60180df410 2024-02-18T11:47:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601001fbc0 con 0x7f60180df410 2024-02-18T11:47:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601804ec40 con 0x7f60180df410 2024-02-18T11:47:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601804f200 con 0x7f60180df410 2024-02-18T11:47:59.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f8148c52640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440b4240 0x7f814414b1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e50 tx=0x7f813c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.075+0000 7f7d676e1640 1 -- 172.21.15.116:0/3190171230 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d600b2110 con 0x7f7d600b25f0 2024-02-18T11:47:59.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f9c3849d640 1 Processor -- start 2024-02-18T11:47:59.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980b93b0 msgr2=0x7f69980ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f699d3b5640 1 --2- 172.21.15.116:0/2998597958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980b93b0 0x7f69980ce7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f69880097a0 tx=0x7f698802b8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273d8ed640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27381450d0 0x7f273815eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273d8ed640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273814daa0 con 0x7f27380c4cc0 2024-02-18T11:47:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273d8ed640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273814dd80 con 0x7f27381450d0 2024-02-18T11:47:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273d8ed640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273814e060 con 0x7f273806d600 2024-02-18T11:47:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f9c3849d640 1 -- start start 2024-02-18T11:47:59.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 shutdown_connections 2024-02-18T11:47:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f699d3b5640 1 --2- 172.21.15.116:0/2998597958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980b93b0 0x7f69980ce7f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 >> 172.21.15.116:0/2998597958 conn(0x7f69980d04a0 msgr2=0x7f69980d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273c8eb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f273806d600 0x7f273813b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273c8eb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f273806d600 0x7f273813b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56476/0 (socket says 172.21.15.116:56476) 2024-02-18T11:47:59.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f273c8eb640 1 -- 172.21.15.116:0/99873992 learned_addr learned my addr 172.21.15.116:0/99873992 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.076+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c02c9f0 con 0x7f81440b4240 2024-02-18T11:47:59.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813c009510 con 0x7f81440b4240 2024-02-18T11:47:59.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813c02c9f0 con 0x7f81440b4240 2024-02-18T11:47:59.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814415e400 con 0x7f81440b4240 2024-02-18T11:47:59.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81441f8230 con 0x7f81440b4240 2024-02-18T11:47:59.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7d5803e8c0 con 0x7f7d600b25f0 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f7d55ffb640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d4406bc70 0x7f7d4406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f2737fff640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27380c4cc0 0x7f273815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f273d0ec640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27381450d0 0x7f273815eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f273c8eb640 1 -- 172.21.15.116:0/99873992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27381450d0 msgr2=0x7f273815eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f273c8eb640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27381450d0 0x7f273815eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f273c8eb640 1 -- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27380c4cc0 msgr2=0x7f273815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7d580ad2c0 con 0x7f7d600b25f0 2024-02-18T11:47:59.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f273c8eb640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27380c4cc0 0x7f273815e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f273c8eb640 1 -- 172.21.15.116:0/99873992 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728009400 con 0x7f273806d600 2024-02-18T11:47:59.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f2737fff640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27380c4cc0 0x7f273815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:59.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f7d55ffb640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] conn(0x7f7d44075370 0x7f7d440777b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 --> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d44077e80 con 0x7f7d44075370 2024-02-18T11:47:59.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7f9c3849d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3006c740 0x7f9c300c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.077+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fb67800abd0 con 0x7fb6800718d0 2024-02-18T11:47:59.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c3849d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c300cf9f0 0x7f9c3015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c3849d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c30161fa0 0x7f9c3015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c3849d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c300ca1e0 con 0x7f9c30161fa0 2024-02-18T11:47:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c3849d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301565b0 con 0x7f9c3006c740 2024-02-18T11:47:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c3849d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301ea710 con 0x7f9c300cf9f0 2024-02-18T11:47:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d58077750 con 0x7f7d600b25f0 2024-02-18T11:47:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3006c740 0x7f9c300c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7fb6677fe640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb66c06bd40 0x7fb66c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f273c8eb640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f273806d600 0x7f273813b360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2728000c00 tx=0x7f27280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3006c740 0x7f9c300c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53998/0 (socket says 172.21.15.116:53998) 2024-02-18T11:47:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 -- 172.21.15.116:0/2542222219 learned_addr learned my addr 172.21.15.116:0/2542222219 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 shutdown_connections 2024-02-18T11:47:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb67808b480 con 0x7fb6800718d0 2024-02-18T11:47:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7fb67effd640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb66c06bd40 0x7fb66c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c35a38640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c300cf9f0 0x7f9c3015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7fb6677fe640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] conn(0x7fb66c0754c0 0x7fb66c077900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 --> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb66c077fd0 con 0x7fb66c0754c0 2024-02-18T11:47:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb67808b830 con 0x7fb6800718d0 2024-02-18T11:47:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f699d3b5640 1 -- 172.21.15.116:0/2998597958 wait complete. 2024-02-18T11:47:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728006e10 con 0x7f273806d600 2024-02-18T11:47:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36a3a640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c30161fa0 0x7f9c3015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272803ad60 con 0x7f273806d600 2024-02-18T11:47:59.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273814e340 con 0x7f273806d600 2024-02-18T11:47:59.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273814cd10 con 0x7f273806d600 2024-02-18T11:47:59.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7fb67f7fe640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] conn(0x7fb66c0754c0 0x7fb66c077900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7fb67effd640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb66c06bd40 0x7fb66c06e200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb670036000 tx=0x7fb670008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 -- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c300cf9f0 msgr2=0x7f9c3015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c300cf9f0 0x7f9c3015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 -- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c30161fa0 msgr2=0x7f9c3015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c30161fa0 0x7f9c3015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.078+0000 7f9c36239640 1 -- 172.21.15.116:0/2542222219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240094d0 con 0x7f9c3006c740 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f9c36239640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3006c740 0x7f9c300c9ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800e9b0 tx=0x7f9c2800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800c9a0 con 0x7f9c3006c740 2024-02-18T11:47:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2800cb00 con 0x7f9c3006c740 2024-02-18T11:47:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c301ea9f0 con 0x7f9c3006c740 2024-02-18T11:47:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800ce90 con 0x7f9c3006c740 2024-02-18T11:47:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f7d65c7e640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] conn(0x7f7d44075370 0x7f7d440777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728044630 con 0x7f273806d600 2024-02-18T11:47:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f7d6547d640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d4406bc70 0x7f7d4406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2718001030 con 0x7f273806d600 2024-02-18T11:47:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f7d65c7e640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] conn(0x7f7d44075370 0x7f7d440777b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f813c03e660 con 0x7f81440b4240 2024-02-18T11:47:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== osd.2 v2:172.21.15.116:6817/3065364614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb92c078000 con 0x7fb92c0754f0 2024-02-18T11:47:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c30156890 con 0x7f9c3006c740 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f7d6547d640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d4406bc70 0x7f7d4406e130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009930 tx=0x7f7d50009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f814406ec70 con 0x7f81440b4240 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.079+0000 7f8140ff9640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f813006bd30 0x7f813006e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f813c0ad770 con 0x7f81440b4240 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f8143fff640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f813006bd30 0x7f813006e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f8140ff9640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] conn(0x7f8130075320 0x7f8130077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 --> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8130077e30 con 0x7f8130075320 2024-02-18T11:47:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f8143fff640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f813006bd30 0x7f813006e1f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8134009620 tx=0x7f8134006b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f813c02c9f0 con 0x7f81440b4240 2024-02-18T11:47:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f2728044790 con 0x7f273806d600 2024-02-18T11:47:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.080+0000 7f699d3b5640 1 Processor -- start 2024-02-18T11:47:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== osd.0 v2:172.21.15.116:6800/991283493 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7d44077e80 con 0x7f7d44075370 2024-02-18T11:47:59.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f600cff9640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60180546c0 con 0x7f60180df410 2024-02-18T11:47:59.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f81437fe640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] conn(0x7f8130075320 0x7f8130077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7fb67f7fe640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] conn(0x7fb66c0754c0 0x7fb66c077900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f601001fd20 con 0x7f60180df410 2024-02-18T11:47:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9c28013470 con 0x7f9c3006c740 2024-02-18T11:47:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f27357fa640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f272406bd40 0x7f272406e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f27280ad220 con 0x7f273806d600 2024-02-18T11:47:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f699d3b5640 1 -- start start 2024-02-18T11:47:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f27357fa640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] conn(0x7f27240754c0 0x7f2724077900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 --> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2724077fd0 con 0x7f27240754c0 2024-02-18T11:47:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.081+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27280ad5d0 con 0x7f273806d600 2024-02-18T11:47:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f2737fff640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f272406bd40 0x7f272406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f81437fe640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] conn(0x7f8130075320 0x7f8130077760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f273d0ec640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] conn(0x7f27240754c0 0x7f2724077900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f9c2effd640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c1c06bc70 0x7f9c1c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9c2808a410 con 0x7f9c3006c740 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f600effd640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f600006bd70 0x7f600006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6010092480 con 0x7f60180df410 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f600effd640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] conn(0x7f60000754e0 0x7f6000077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6000077ff0 con 0x7f60000754e0 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60100949c0 con 0x7f60180df410 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f9c3849d640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] conn(0x7f9c30155880 0x7f9c30155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 --> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c301eaf10 con 0x7f9c30155880 2024-02-18T11:47:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f601dd9b640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] conn(0x7f60000754e0 0x7f6000077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f273d0ec640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] conn(0x7f27240754c0 0x7f2724077900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== osd.6 v2:172.21.15.143:6800/3882163910 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb66c077fd0 con 0x7fb66c0754c0 2024-02-18T11:47:59.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f601e59c640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f600006bd70 0x7f600006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f699d3b5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69980b93b0 0x7f69981fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f699d3b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980ced30 0x7f699815f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f699d3b5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f699815dba0 0x7f69980b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f699d3b5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69981fc470 con 0x7f69980ced30 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f699d3b5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69980c8580 con 0x7f69980b93b0 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f699d3b5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69981f8430 con 0x7f699815dba0 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69967fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980ced30 0x7f699815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69967fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980ced30 0x7f699815f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57326/0 (socket says 172.21.15.116:57326) 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69967fc640 1 -- 172.21.15.116:0/1929989643 learned_addr learned my addr 172.21.15.116:0/1929989643 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69977fe640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f699815dba0 0x7f69980b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f9c35a38640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c1c06bc70 0x7f9c1c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.082+0000 7f9c36a3a640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] conn(0x7f9c30155880 0x7f9c30155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f9c36a3a640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] conn(0x7f9c30155880 0x7f9c30155c60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== osd.5 v2:172.21.15.143:6824/680136748 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8130077e30 con 0x7f8130075320 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f6996ffd640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69980b93b0 0x7f69981fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69977fe640 1 -- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69980b93b0 msgr2=0x7f69981fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69977fe640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69980b93b0 0x7f69981fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69977fe640 1 -- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980ced30 msgr2=0x7f699815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69977fe640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69980ced30 0x7f699815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f69977fe640 1 -- 172.21.15.116:0/1929989643 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988009400 con 0x7f699815dba0 2024-02-18T11:47:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f601e59c640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f600006bd70 0x7f600006e230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f601804f530 tx=0x7f6014006b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.083+0000 7f601dd9b640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] conn(0x7f60000754e0 0x7f6000077920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f69977fe640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f699815dba0 0x7f69980b7cb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f699000be20 tx=0x7f699000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f9c35a38640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c1c06bc70 0x7f9c1c06e130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c300cf6b0 tx=0x7f9c20007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== osd.4 v2:172.21.15.143:6802/2574593080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9c301eaf10 con 0x7f9c30155880 2024-02-18T11:47:59.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== osd.3 v2:172.21.15.116:6816/1865949701 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2724077fd0 con 0x7f27240754c0 2024-02-18T11:47:59.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f2737fff640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f272406bd40 0x7f272406e200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2738140be0 tx=0x7f2730009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.085+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== osd.1 v2:172.21.15.116:6808/208468530 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6000077ff0 con 0x7f60000754e0 2024-02-18T11:47:59.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699000ce70 con 0x7f699815dba0 2024-02-18T11:47:59.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6990014440 con 0x7f699815dba0 2024-02-18T11:47:59.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6990014810 con 0x7f699815dba0 2024-02-18T11:47:59.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69981f8710 con 0x7f699815dba0 2024-02-18T11:47:59.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.084+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69981f8cd0 con 0x7f699815dba0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.086+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6978001030 con 0x7f699815dba0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.089+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f699001d3b0 con 0x7f699815dba0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.089+0000 7f697f7fe640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f698406bd40 0x7f698406e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.089+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f699008b6e0 con 0x7f699815dba0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.089+0000 7f697f7fe640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] conn(0x7f69840754c0 0x7f6984077900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.090+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 --> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6984077fd0 con 0x7f69840754c0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.090+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f699008ba90 con 0x7f699815dba0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.090+0000 7f6996ffd640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f698406bd40 0x7f698406e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.091+0000 7f6996ffd640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f698406bd40 0x7f698406e200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f698c000f60 tx=0x7f698c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.091+0000 7f69967fc640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] conn(0x7f69840754c0 0x7f6984077900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.091+0000 7f69967fc640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] conn(0x7f69840754c0 0x7f6984077900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.093+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== osd.7 v2:172.21.15.143:6816/374457617 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6984077fd0 con 0x7f69840754c0 2024-02-18T11:47:59.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.107+0000 7fb94e23b640 1 -- 172.21.15.116:0/3010974406 --> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9400364c0 con 0x7fb92c0754f0 2024-02-18T11:47:59.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.112+0000 7fb93dffb640 1 -- 172.21.15.116:0/3010974406 <== osd.2 v2:172.21.15.116:6817/3065364614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb93800a380 con 0x7fb92c0754f0 2024-02-18T11:47:59.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.112+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] conn(0x7fb92c0754f0 msgr2=0x7fb92c077930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.112+0000 7fb92b7fe640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6817/3065364614,v1:172.21.15.116:6819/3065364614] conn(0x7fb92c0754f0 0x7fb92c077930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb92c06bd70 msgr2=0x7fb92c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb92b7fe640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb92c06bd70 0x7fb92c06e230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb944036000 tx=0x7fb94403c040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940048c30 msgr2=0x7fb94017dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb92b7fe640 1 --2- 172.21.15.116:0/3010974406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb940048c30 0x7fb94017dd70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb94802d370 tx=0x7fb948068690 comp rx=0 tx=0).stop 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb94da3a640 1 -- 172.21.15.116:0/3010974406 reap_dead start 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 shutdown_connections 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 >> 172.21.15.116:0/3010974406 conn(0x7fb94001ef30 msgr2=0x7fb940037520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.113+0000 7f7d676e1640 1 -- 172.21.15.116:0/3190171230 --> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d600b2db0 con 0x7f7d44075370 2024-02-18T11:47:59.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.114+0000 7f7d55ffb640 1 -- 172.21.15.116:0/3190171230 <== osd.0 v2:172.21.15.116:6800/991283493 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7d5c00c040 con 0x7f7d44075370 2024-02-18T11:47:59.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.114+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 shutdown_connections 2024-02-18T11:47:59.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.114+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] conn(0x7f7d44075370 msgr2=0x7f7d440777b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.114+0000 7fb92b7fe640 1 -- 172.21.15.116:0/3010974406 wait complete. 2024-02-18T11:47:59.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.114+0000 7f7d437fe640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6800/991283493,v1:172.21.15.116:6801/991283493] conn(0x7f7d44075370 0x7f7d440777b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.115+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d4406bc70 msgr2=0x7f7d4406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.115+0000 7f7d437fe640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d4406bc70 0x7f7d4406e130 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009930 tx=0x7f7d50009190 comp rx=0 tx=0).stop 2024-02-18T11:47:59.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.115+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b25f0 msgr2=0x7f7d601596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.115+0000 7f7d437fe640 1 --2- 172.21.15.116:0/3190171230 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d600b25f0 0x7f7d601596c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7d58032000 tx=0x7f7d58004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:59.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.115+0000 7f7d65c7e640 1 -- 172.21.15.116:0/3190171230 reap_dead start 2024-02-18T11:47:59.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 shutdown_connections 2024-02-18T11:47:59.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 --> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8144159490 con 0x7f8130075320 2024-02-18T11:47:59.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.115+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 --> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb680154520 con 0x7fb66c0754c0 2024-02-18T11:47:59.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 >> 172.21.15.116:0/3190171230 conn(0x7f7d60067000 msgr2=0x7f7d6006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 --> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2718002d30 con 0x7f27240754c0 2024-02-18T11:47:59.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 shutdown_connections 2024-02-18T11:47:59.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f7d437fe640 1 -- 172.21.15.116:0/3190171230 wait complete. 2024-02-18T11:47:59.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f8140ff9640 1 -- 172.21.15.116:0/2109945141 <== osd.5 v2:172.21.15.143:6824/680136748 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8138001830 con 0x7f8130075320 2024-02-18T11:47:59.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7fb6677fe640 1 -- 172.21.15.116:0/645835164 <== osd.6 v2:172.21.15.143:6800/3882163910 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb6740058b0 con 0x7fb66c0754c0 2024-02-18T11:47:59.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.116+0000 7f27357fa640 1 -- 172.21.15.116:0/99873992 <== osd.3 v2:172.21.15.116:6816/1865949701 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2718002d30 con 0x7f27240754c0 2024-02-18T11:47:59.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 --> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9c3015e880 con 0x7f9c30155880 2024-02-18T11:47:59.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] conn(0x7f8130075320 msgr2=0x7f8130077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:6824/680136748,v1:172.21.15.143:6825/680136748] conn(0x7f8130075320 0x7f8130077760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] conn(0x7fb66c0754c0 msgr2=0x7fb66c077900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.143:6800/3882163910,v1:172.21.15.143:6801/3882163910] conn(0x7fb66c0754c0 0x7fb66c077900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb66c06bd40 msgr2=0x7fb66c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb66c06bd40 0x7fb66c06e200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb670036000 tx=0x7fb670008030 comp rx=0 tx=0).stop 2024-02-18T11:47:59.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6800718d0 msgr2=0x7fb68014a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 --2- 172.21.15.116:0/645835164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6800718d0 0x7fb68014a430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb67800c800 tx=0x7fb67800ccc0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f813006bd30 msgr2=0x7f813006e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f813006bd30 0x7f813006e1f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8134009620 tx=0x7f8134006b30 comp rx=0 tx=0).stop 2024-02-18T11:47:59.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f9c2effd640 1 -- 172.21.15.116:0/2542222219 <== osd.4 v2:172.21.15.143:6802/2574593080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9c3015e880 con 0x7f9c30155880 2024-02-18T11:47:59.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440b4240 msgr2=0x7f814414b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb67f7fe640 1 -- 172.21.15.116:0/645835164 reap_dead start 2024-02-18T11:47:59.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 --2- 172.21.15.116:0/2109945141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81440b4240 0x7f814414b1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e50 tx=0x7f813c004a40 comp rx=0 tx=0).stop 2024-02-18T11:47:59.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f8148c52640 1 -- 172.21.15.116:0/2109945141 reap_dead start 2024-02-18T11:47:59.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 shutdown_connections 2024-02-18T11:47:59.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 >> 172.21.15.116:0/645835164 conn(0x7fb680067000 msgr2=0x7fb6800aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 shutdown_connections 2024-02-18T11:47:59.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7fb6657fa640 1 -- 172.21.15.116:0/645835164 wait complete. 2024-02-18T11:47:59.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 shutdown_connections 2024-02-18T11:47:59.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.117+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 >> 172.21.15.116:0/2109945141 conn(0x7f8144067000 msgr2=0x7f81440711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] conn(0x7f9c30155880 msgr2=0x7f9c30155c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:6802/2574593080,v1:172.21.15.143:6803/2574593080] conn(0x7f9c30155880 0x7f9c30155c60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 --> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6978002d30 con 0x7f69840754c0 2024-02-18T11:47:59.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 shutdown_connections 2024-02-18T11:47:59.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f814a6b5640 1 -- 172.21.15.116:0/2109945141 wait complete. 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] conn(0x7f27240754c0 msgr2=0x7f2724077900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6816/1865949701,v1:172.21.15.116:6818/1865949701] conn(0x7f27240754c0 0x7f2724077900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f272406bd40 msgr2=0x7f272406e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f272406bd40 0x7f272406e200 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2738140be0 tx=0x7f2730009040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f273806d600 msgr2=0x7f273813b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 --2- 172.21.15.116:0/99873992 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f273806d600 0x7f273813b360 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2728000c00 tx=0x7f27280046c0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c1c06bc70 msgr2=0x7f9c1c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9c1c06bc70 0x7f9c1c06e130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c300cf6b0 tx=0x7f9c20007240 comp rx=0 tx=0).stop 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3006c740 msgr2=0x7f9c300c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 --2- 172.21.15.116:0/2542222219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c3006c740 0x7f9c300c9ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800e9b0 tx=0x7f9c2800ee70 comp rx=0 tx=0).stop 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c36a3a640 1 -- 172.21.15.116:0/2542222219 reap_dead start 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 shutdown_connections 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 >> 172.21.15.116:0/2542222219 conn(0x7f9c300d3a20 msgr2=0x7f9c301630f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f697f7fe640 1 -- 172.21.15.116:0/1929989643 <== osd.7 v2:172.21.15.143:6816/374457617 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6978002d30 con 0x7f69840754c0 2024-02-18T11:47:59.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 shutdown_connections 2024-02-18T11:47:59.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d0ec640 1 -- 172.21.15.116:0/99873992 reap_dead start 2024-02-18T11:47:59.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 shutdown_connections 2024-02-18T11:47:59.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.118+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 >> 172.21.15.116:0/99873992 conn(0x7f2738066f20 msgr2=0x7f27380b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 shutdown_connections 2024-02-18T11:47:59.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f273d8ed640 1 -- 172.21.15.116:0/99873992 wait complete. 2024-02-18T11:47:59.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f9c3849d640 1 -- 172.21.15.116:0/2542222219 wait complete. 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] conn(0x7f69840754c0 msgr2=0x7f6984077900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f699d3b5640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:6816/374457617,v1:172.21.15.143:6817/374457617] conn(0x7f69840754c0 0x7f6984077900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f698406bd40 msgr2=0x7f698406e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f699d3b5640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f698406bd40 0x7f698406e200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f698c000f60 tx=0x7f698c009040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f699815dba0 msgr2=0x7f69980b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.119+0000 7f699d3b5640 1 --2- 172.21.15.116:0/1929989643 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f699815dba0 0x7f69980b7cb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f699000be20 tx=0x7f699000bf10 comp rx=0 tx=0).stop 2024-02-18T11:47:59.170 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.120+0000 7f69977fe640 1 -- 172.21.15.116:0/1929989643 reap_dead start 2024-02-18T11:47:59.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.120+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 shutdown_connections 2024-02-18T11:47:59.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.120+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 >> 172.21.15.116:0/1929989643 conn(0x7f69980d04a0 msgr2=0x7f6998161540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.120+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 shutdown_connections 2024-02-18T11:47:59.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.120+0000 7f699d3b5640 1 -- 172.21.15.116:0/1929989643 wait complete. 2024-02-18T11:47:59.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.120+0000 7f600cff9640 1 -- 172.21.15.116:0/1856450914 --> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6018048450 con 0x7f60000754e0 2024-02-18T11:47:59.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.121+0000 7f600effd640 1 -- 172.21.15.116:0/1856450914 <== osd.1 v2:172.21.15.116:6808/208468530 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6018048450 con 0x7f60000754e0 2024-02-18T11:47:59.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.122+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] conn(0x7f60000754e0 msgr2=0x7f6000077920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.122+0000 7f6024965640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6808/208468530,v1:172.21.15.116:6809/208468530] conn(0x7f60000754e0 0x7f6000077920 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.122+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f600006bd70 msgr2=0x7f600006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.122+0000 7f6024965640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f600006bd70 0x7f600006e230 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f601804f530 tx=0x7f6014006b30 comp rx=0 tx=0).stop 2024-02-18T11:47:59.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.122+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60180df410 msgr2=0x7f60180e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.122+0000 7f6024965640 1 --2- 172.21.15.116:0/1856450914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60180df410 0x7f60180e0cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f601000ab00 tx=0x7f6010007170 comp rx=0 tx=0).stop 2024-02-18T11:47:59.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.123+0000 7f601ed9d640 1 -- 172.21.15.116:0/1856450914 reap_dead start 2024-02-18T11:47:59.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.123+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 shutdown_connections 2024-02-18T11:47:59.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.123+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 >> 172.21.15.116:0/1856450914 conn(0x7f6018009fe0 msgr2=0x7f60180342e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.123+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 shutdown_connections 2024-02-18T11:47:59.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.123+0000 7f6024965640 1 -- 172.21.15.116:0/1856450914 wait complete. 2024-02-18T11:47:59.174 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.174 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-18T11:47:59.175 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.175 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-18T11:47:59.176 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.176 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-18T11:47:59.177 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.177 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-18T11:47:59.178 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.178 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-18T11:47:59.179 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.179 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-18T11:47:59.180 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.180 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-18T11:47:59.181 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:47:59.182 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-18T11:47:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.390+0000 7f5ca150d640 1 Processor -- start 2024-02-18T11:47:59.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.390+0000 7f5ca150d640 1 -- start start 2024-02-18T11:47:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.391+0000 7f78647d0640 1 Processor -- start 2024-02-18T11:47:59.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.391+0000 7f78647d0640 1 -- start start 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5ca150d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c9c0b1550 0x7f5c9c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5ca150d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5c9c06ab60 0x7f5c9c1661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5ca150d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5ca150d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c06b5b0 con 0x7f5c9c0b1550 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5ca150d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c165d50 con 0x7f5c9c06ab60 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5ca150d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0afaa0 con 0x7f5c9c0ccce0 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.391+0000 7f87056c9640 1 Processor -- start 2024-02-18T11:47:59.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.391+0000 7f87056c9640 1 -- start start 2024-02-18T11:47:59.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5c9affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c9c0b1550 0x7f5c9c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7ff4c0a36640 1 Processor -- start 2024-02-18T11:47:59.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5c9affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c9c0b1550 0x7f5c9c0b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57340/0 (socket says 172.21.15.116:57340) 2024-02-18T11:47:59.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7f5c9affd640 1 -- 172.21.15.116:0/3818109515 learned_addr learned my addr 172.21.15.116:0/3818109515 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7fa1fcfec640 1 Processor -- start 2024-02-18T11:47:59.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7fa1fcfec640 1 -- start start 2024-02-18T11:47:59.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.392+0000 7ff4c0a36640 1 -- start start 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f78647d0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78540e12a0 0x7f78540534e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f78647d0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854053a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f78647d0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7854054fd0 con 0x7f7854054b40 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f78647d0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7854055c50 con 0x7f78540e12a0 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f78647d0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78540552b0 con 0x7f78540e4d20 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f5c9b7fe640 1 --2- 172.21.15.116:0/3818109515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f7861d6b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7854054b40 0x7f78540e1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45948/0 (socket says 172.21.15.116:45948) 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f7861d6b640 1 -- 172.21.15.116:0/2896234545 learned_addr learned my addr 172.21.15.116:0/2896234545 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f786256c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78540e12a0 0x7f78540534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f7862d6d640 1 --2- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854053a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f5c9affd640 1 -- 172.21.15.116:0/3818109515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 msgr2=0x7f5c9c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f5c9affd640 1 --2- 172.21.15.116:0/3818109515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c0d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f5c9affd640 1 -- 172.21.15.116:0/3818109515 >> v2:172.21.15.143:3300/0 conn(0x7f5c9c06ab60 msgr2=0x7f5c9c1661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f5c9affd640 1 --2- 172.21.15.116:0/3818109515 >> v2:172.21.15.143:3300/0 conn(0x7f5c9c06ab60 0x7f5c9c1661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.393+0000 7f5c9affd640 1 -- 172.21.15.116:0/3818109515 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0b1e70 con 0x7f5c9c0b1550 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f786256c640 1 -- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 msgr2=0x7f7854053a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f786256c640 1 --2- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854053a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f786256c640 1 -- 172.21.15.116:0/2896234545 >> v1:172.21.15.116:6789/0 conn(0x7f7854054b40 legacy=0x7f78540e1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f786256c640 1 -- 172.21.15.116:0/2896234545 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854055590 con 0x7f78540e12a0 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1fcfec640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa1f8147ee0 0x7fa1f8071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1fcfec640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 0x7fa1f80b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1fcfec640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f80cb950 con 0x7fa1f8165cf0 2024-02-18T11:47:59.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1fcfec640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f8166290 con 0x7fa1f8147ee0 2024-02-18T11:47:59.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1fcfec640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8166570 con 0x7fa1f81658b0 2024-02-18T11:47:59.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4c0a36640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff4b40085d0 0x7ff4b40dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4c0a36640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff4b40e42d0 0x7ff4b4006ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4c0a36640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40094b0 con 0x7ff4b40089b0 2024-02-18T11:47:59.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4c0a36640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b4056590 con 0x7ff4b40085d0 2024-02-18T11:47:59.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4c0a36640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b4056870 con 0x7ff4b40e42d0 2024-02-18T11:47:59.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1f6ffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 0x7fa1f80b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1f6ffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 0x7fa1f80b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56518/0 (socket says 172.21.15.116:56518) 2024-02-18T11:47:59.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1f6ffd640 1 -- 172.21.15.116:0/1961051914 learned_addr learned my addr 172.21.15.116:0/1961051914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4bbfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff4b40e42d0 0x7ff4b4006ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7fa1f77fe640 1 --2- 172.21.15.116:0/1961051914 >> v2:172.21.15.143:3300/0 conn(0x7fa1f8147ee0 0x7fa1f8071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4bbfff640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff4b40e42d0 0x7ff4b4006ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:54040/0 (socket says 172.21.15.116:54040) 2024-02-18T11:47:59.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4bbfff640 1 -- 172.21.15.116:0/3358069754 learned_addr learned my addr 172.21.15.116:0/3358069754 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7ff4bb7fe640 1 --2- 172.21.15.116:0/3358069754 >> v2:172.21.15.116:3300/0 conn(0x7ff4b40085d0 0x7ff4b40dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.394 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f434f160640 1 Processor -- start 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f87056c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 0x7f86f8054cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f86feffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 0x7f86f8054cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f86feffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 0x7f86f8054cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56516/0 (socket says 172.21.15.116:56516) 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f87056c9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f8053c20 con 0x7f86f8050080 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f86feffd640 1 -- 172.21.15.116:0/179192636 learned_addr learned my addr 172.21.15.116:0/179192636 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f8053f00 con 0x7f86f803ad40 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.394+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80dfac0 con 0x7f86f80df6e0 2024-02-18T11:47:59.395 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7fa1f6ffd640 1 -- 172.21.15.116:0/1961051914 >> v2:172.21.15.143:3300/0 conn(0x7fa1f8147ee0 msgr2=0x7fa1f8071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7fa1f6ffd640 1 --2- 172.21.15.116:0/1961051914 >> v2:172.21.15.143:3300/0 conn(0x7fa1f8147ee0 0x7fa1f8071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7fa1f6ffd640 1 -- 172.21.15.116:0/1961051914 >> v1:172.21.15.116:6789/0 conn(0x7fa1f8165cf0 legacy=0x7fa1f80cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f5c9affd640 1 --2- 172.21.15.116:0/3818109515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b1550 0x7f5c9c0b1930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c06bc40 tx=0x7f5c9002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bbd83e35f4f135 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f86fdffb640 1 -- 172.21.15.116:0/179192636 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655842251 0 0) 0x7f86f8053f00 con 0x7f86f803ad40 2024-02-18T11:47:59.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f86fdffb640 1 -- 172.21.15.116:0/179192636 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e8003440 con 0x7f86f803ad40 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f7139353640 1 Processor -- start 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff53e72c640 1 Processor -- start 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff4ba7fc640 1 -- 172.21.15.116:0/3358069754 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864476444 0 0) 0x7ff4b40094b0 con 0x7ff4b40089b0 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff4ba7fc640 1 -- 172.21.15.116:0/3358069754 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ac003520 con 0x7ff4b40089b0 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff4bbfff640 1 -- 172.21.15.116:0/3358069754 >> v2:172.21.15.116:3300/0 conn(0x7ff4b40085d0 msgr2=0x7ff4b40dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff4bbfff640 1 --2- 172.21.15.116:0/3358069754 >> v2:172.21.15.116:3300/0 conn(0x7ff4b40085d0 0x7ff4b40dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff4bbfff640 1 -- 172.21.15.116:0/3358069754 >> v1:172.21.15.116:6789/0 conn(0x7ff4b40089b0 legacy=0x7ff4b40e3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff4bbfff640 1 -- 172.21.15.116:0/3358069754 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4008e60 con 0x7ff4b40e42d0 2024-02-18T11:47:59.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f434f160640 1 -- start start 2024-02-18T11:47:59.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7fa1f6ffd640 1 -- 172.21.15.116:0/1961051914 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80cbc10 con 0x7fa1f81658b0 2024-02-18T11:47:59.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7ff53e72c640 1 -- start start 2024-02-18T11:47:59.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f86feffd640 1 -- 172.21.15.116:0/179192636 >> v1:172.21.15.143:6789/0 conn(0x7f86f803ad40 legacy=0x7f86f80533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f86feffd640 1 -- 172.21.15.116:0/179192636 >> v1:172.21.15.116:6789/0 conn(0x7f86f8050080 legacy=0x7f86f80384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f86feffd640 1 -- 172.21.15.116:0/179192636 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f80541e0 con 0x7f86f80df6e0 2024-02-18T11:47:59.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f5c99ffb640 1 -- 172.21.15.116:0/3818109515 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90031880 con 0x7f5c9c0b1550 2024-02-18T11:47:59.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f5c99ffb640 1 -- 172.21.15.116:0/3818109515 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c900319e0 con 0x7f5c9c0b1550 2024-02-18T11:47:59.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f5c99ffb640 1 -- 172.21.15.116:0/3818109515 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90031d70 con 0x7f5c9c0b1550 2024-02-18T11:47:59.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f786256c640 1 --2- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e12a0 0x7f78540534e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78580097a0 tx=0x7f785802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573cd378d7cd6c0f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f786156a640 1 -- 172.21.15.116:0/2896234545 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858031830 con 0x7f78540e12a0 2024-02-18T11:47:59.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f786156a640 1 -- 172.21.15.116:0/2896234545 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7858031990 con 0x7f78540e12a0 2024-02-18T11:47:59.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f786156a640 1 -- 172.21.15.116:0/2896234545 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858031d20 con 0x7f78540e12a0 2024-02-18T11:47:59.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.395+0000 7f7139353640 1 -- start start 2024-02-18T11:47:59.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f7139353640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7134162020 0x7f71340ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f7139353640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f713406e7a0 0x7f71340c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7fa1f6ffd640 1 --2- 172.21.15.116:0/1961051914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 0x7fa1f80b25b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009e50 tx=0x7fa1f002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc8a5582e9c2232a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff4bbfff640 1 --2- 172.21.15.116:0/3358069754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b4006ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc02d370 tx=0x7ff4bc0a6960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48fc9d09dd62d45b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53e72c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5381622d0 0x7ff5380b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53e72c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 0x7ff5380d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53e72c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5380ceda0 con 0x7ff538162ca0 2024-02-18T11:47:59.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53e72c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5380b5bd0 con 0x7ff5381622d0 2024-02-18T11:47:59.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53e72c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380cefe0 con 0x7ff5380b5780 2024-02-18T11:47:59.402 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f434f160640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340054b50 0x7f4340051af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff4ba7fc640 1 -- 172.21.15.116:0/3358069754 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0ab6d0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f5ca150d640 1 -- 172.21.15.116:0/3818109515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b1550 msgr2=0x7f5c9c0b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f5ca150d640 1 --2- 172.21.15.116:0/3818109515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b1550 0x7f5c9c0b1930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c06bc40 tx=0x7f5c9002b920 comp rx=0 tx=0).stop 2024-02-18T11:47:59.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f71337fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f713406e7a0 0x7f71340c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f71337fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f713406e7a0 0x7f71340c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57358/0 (socket says 172.21.15.116:57358) 2024-02-18T11:47:59.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff4ba7fc640 1 -- 172.21.15.116:0/3358069754 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc0ab870 con 0x7ff4b40e42d0 2024-02-18T11:47:59.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f86feffd640 1 --2- 172.21.15.116:0/179192636 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 0x7f86f8054cc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f86f40098a0 tx=0x7f86f402b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61475781a0a2b3d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f71337fe640 1 -- 172.21.15.116:0/3104888105 learned_addr learned my addr 172.21.15.116:0/3104888105 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7fa1f67fc640 1 -- 172.21.15.116:0/1961051914 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f002cb00 con 0x7fa1f81658b0 2024-02-18T11:47:59.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53ccc9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 0x7ff5380d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7ff53ccc9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 0x7ff5380d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56532/0 (socket says 172.21.15.116:56532) 2024-02-18T11:47:59.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff53ccc9640 1 -- 172.21.15.116:0/2316313365 learned_addr learned my addr 172.21.15.116:0/2316313365 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff4ba7fc640 1 -- 172.21.15.116:0/3358069754 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0abca0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 -- 172.21.15.116:0/3818109515 shutdown_connections 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f434f160640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43400e26a0 con 0x7f43400e12b0 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f434f160640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4340054fe0 con 0x7f4340053c00 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.396+0000 7f434f160640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4340055c60 con 0x7f4340054b50 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 --2- 172.21.15.116:0/3818109515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 --2- 172.21.15.116:0/3818109515 >> v2:172.21.15.143:3300/0 conn(0x7f5c9c06ab60 0x7f5c9c1661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 --2- 172.21.15.116:0/3818109515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b1550 0x7f5c9c0b1930 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c06bc40 tx=0x7f5c9002b920 comp rx=0 tx=0).stop 2024-02-18T11:47:59.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 -- 172.21.15.116:0/3818109515 >> 172.21.15.116:0/3818109515 conn(0x7f5c9c072630 msgr2=0x7f5c9c099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff5377fe640 1 --2- 172.21.15.116:0/2316313365 >> v2:172.21.15.116:3300/0 conn(0x7ff5381622d0 0x7ff5380b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7fa1f67fc640 1 -- 172.21.15.116:0/1961051914 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f002cc60 con 0x7fa1f81658b0 2024-02-18T11:47:59.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 -- 172.21.15.116:0/3818109515 shutdown_connections 2024-02-18T11:47:59.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f5ca150d640 1 -- 172.21.15.116:0/3818109515 wait complete. 2024-02-18T11:47:59.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff53ccc9640 1 -- 172.21.15.116:0/2316313365 >> v2:172.21.15.116:3300/0 conn(0x7ff5381622d0 msgr2=0x7ff5380b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 -- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e12a0 msgr2=0x7f78540534e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 --2- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e12a0 0x7f78540534e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78580097a0 tx=0x7f785802b8d0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f434e15e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f43400e12b0 0x7f43400534f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45984/0 (socket says 172.21.15.116:45984) 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f434e15e640 1 -- 172.21.15.116:0/2999097958 learned_addr learned my addr 172.21.15.116:0/2999097958 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 -- 172.21.15.116:0/2896234545 shutdown_connections 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 --2- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854053a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 --2- 172.21.15.116:0/2896234545 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e12a0 0x7f78540534e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 -- 172.21.15.116:0/2896234545 >> 172.21.15.116:0/2896234545 conn(0x7f7854020030 msgr2=0x7f7854056d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f7133fff640 1 --2- 172.21.15.116:0/3104888105 >> v2:172.21.15.143:3300/0 conn(0x7f7134162020 0x7f71340ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 -- 172.21.15.116:0/2896234545 shutdown_connections 2024-02-18T11:47:59.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340ad8d0 con 0x7f713406ebe0 2024-02-18T11:47:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f78647d0640 1 -- 172.21.15.116:0/2896234545 wait complete. 2024-02-18T11:47:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f86fdffb640 1 -- 172.21.15.116:0/179192636 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4031870 con 0x7f86f80df6e0 2024-02-18T11:47:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f86fdffb640 1 -- 172.21.15.116:0/179192636 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f40319d0 con 0x7f86f80df6e0 2024-02-18T11:47:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f86fdffb640 1 -- 172.21.15.116:0/179192636 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4031d40 con 0x7f86f80df6e0 2024-02-18T11:47:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 msgr2=0x7f86f8054cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f87056c9640 1 --2- 172.21.15.116:0/179192636 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 0x7f86f8054cc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f86f40098a0 tx=0x7f86f402b910 comp rx=0 tx=0).stop 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713406fa20 con 0x7f713406e7a0 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340cec00 con 0x7f7134162020 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330099516 0 0) 0x7f71340ad8d0 con 0x7f713406ebe0 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c003520 con 0x7f713406ebe0 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727761544 0 0) 0x7f711c003520 con 0x7f713406ebe0 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 >> v2:172.21.15.143:3300/0 conn(0x7f7134162020 msgr2=0x7f71340ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 --2- 172.21.15.116:0/3104888105 >> v2:172.21.15.143:3300/0 conn(0x7f7134162020 0x7f71340ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 >> v2:172.21.15.116:3300/0 conn(0x7f713406e7a0 msgr2=0x7f71340c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 --2- 172.21.15.116:0/3104888105 >> v2:172.21.15.116:3300/0 conn(0x7f713406e7a0 0x7f71340c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713406f180 con 0x7f713406ebe0 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7fa1f67fc640 1 -- 172.21.15.116:0/1961051914 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f00343f0 con 0x7fa1f81658b0 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1961051914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 msgr2=0x7fa1f80b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1961051914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 0x7fa1f80b25b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009e50 tx=0x7fa1f002bf60 comp rx=0 tx=0).stop 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1961051914 shutdown_connections 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1961051914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f81658b0 0x7fa1f80b25b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1961051914 >> v2:172.21.15.143:3300/0 conn(0x7fa1f8147ee0 0x7fa1f8071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1961051914 >> 172.21.15.116:0/1961051914 conn(0x7fa1f80b12b0 msgr2=0x7fa1f80b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff53ccc9640 1 --2- 172.21.15.116:0/2316313365 >> v2:172.21.15.116:3300/0 conn(0x7ff5381622d0 0x7ff5380b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff53ccc9640 1 -- 172.21.15.116:0/2316313365 >> v1:172.21.15.143:6789/0 conn(0x7ff538162ca0 legacy=0x7ff5380ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7ff53ccc9640 1 -- 172.21.15.116:0/2316313365 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380cf2c0 con 0x7ff5380b5780 2024-02-18T11:47:59.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff5377fe640 1 --2- 172.21.15.116:0/2316313365 >> v2:172.21.15.116:3300/0 conn(0x7ff5381622d0 0x7ff5380b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff4c0a36640 1 -- 172.21.15.116:0/3358069754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 msgr2=0x7ff4b4006ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff4c0a36640 1 --2- 172.21.15.116:0/3358069754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b4006ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc02d370 tx=0x7ff4bc0a6960 comp rx=0 tx=0).stop 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff4c0a36640 1 -- 172.21.15.116:0/3358069754 shutdown_connections 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff4c0a36640 1 --2- 172.21.15.116:0/3358069754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b4006ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff4c0a36640 1 --2- 172.21.15.116:0/3358069754 >> v2:172.21.15.116:3300/0 conn(0x7ff4b40085d0 0x7ff4b40dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff4c0a36640 1 -- 172.21.15.116:0/3358069754 >> 172.21.15.116:0/3358069754 conn(0x7ff4b401ef40 msgr2=0x7ff4b4055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f434d95d640 1 --2- 172.21.15.116:0/2999097958 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340054b50 0x7f4340051af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f434d95d640 1 -- 172.21.15.116:0/2999097958 >> v1:172.21.15.143:6789/0 conn(0x7f4340053c00 legacy=0x7f4340052030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f434d95d640 1 -- 172.21.15.116:0/2999097958 >> v1:172.21.15.116:6789/0 conn(0x7f43400e12b0 legacy=0x7f43400534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.397+0000 7f434d95d640 1 -- 172.21.15.116:0/2999097958 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340009be0 con 0x7f4340054b50 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f712c0019f0 con 0x7f713406ebe0 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 >> v1:172.21.15.116:6789/0 conn(0x7f713406ebe0 legacy=0x7f71340cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f711c0039c0 0x7f711c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c0047c0 0x7f711c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c005180 0x7f711c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713406fa20 con 0x7f711c0039c0 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340cec00 con 0x7f711c005180 2024-02-18T11:47:59.417 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c00c0b0 con 0x7f711c0047c0 2024-02-18T11:47:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1961051914 shutdown_connections 2024-02-18T11:47:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f71337fe640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c0047c0 0x7f711c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7138b52640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c005180 0x7f711c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7ff53ccc9640 1 --2- 172.21.15.116:0/2316313365 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 0x7ff5380d1950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff528009870 tx=0x7ff52802b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455d0d852ad24531 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7f7133fff640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f711c0039c0 0x7f711c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7ff536ffd640 1 -- 172.21.15.116:0/2316313365 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528030860 con 0x7ff5380b5780 2024-02-18T11:47:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7ff536ffd640 1 -- 172.21.15.116:0/2316313365 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5280309c0 con 0x7ff5380b5780 2024-02-18T11:47:59.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f7138b52640 1 -- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c0047c0 msgr2=0x7f711c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f7138b52640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c0047c0 0x7f711c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f7138b52640 1 -- 172.21.15.116:0/3104888105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f711c0039c0 msgr2=0x7f711c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:47:59.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f7138b52640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f711c0039c0 0x7f711c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f5ca150d640 1 Processor -- start 2024-02-18T11:47:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f7138b52640 1 -- 172.21.15.116:0/3104888105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c003520 con 0x7f711c005180 2024-02-18T11:47:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f78647d0640 1 Processor -- start 2024-02-18T11:47:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f78647d0640 1 -- start start 2024-02-18T11:47:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7ff4c0a36640 1 -- 172.21.15.116:0/3358069754 shutdown_connections 2024-02-18T11:47:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 shutdown_connections 2024-02-18T11:47:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f87056c9640 1 --2- 172.21.15.116:0/179192636 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f80df6e0 0x7f86f8054cc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 >> 172.21.15.116:0/179192636 conn(0x7f86f8055340 msgr2=0x7f86f8055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7ff4c0a36640 1 -- 172.21.15.116:0/3358069754 wait complete. 2024-02-18T11:47:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 shutdown_connections 2024-02-18T11:47:59.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f87056c9640 1 -- 172.21.15.116:0/179192636 wait complete. 2024-02-18T11:47:59.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7ff536ffd640 1 -- 172.21.15.116:0/2316313365 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528030d90 con 0x7ff5380b5780 2024-02-18T11:47:59.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 -- 172.21.15.116:0/2316313365 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 msgr2=0x7ff5380d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 --2- 172.21.15.116:0/2316313365 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 0x7ff5380d1950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff528009870 tx=0x7ff52802b9a0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f7138b52640 1 -- 172.21.15.116:0/3104888105 reap_dead start 2024-02-18T11:47:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 -- 172.21.15.116:0/2316313365 shutdown_connections 2024-02-18T11:47:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 --2- 172.21.15.116:0/2316313365 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5380b5780 0x7ff5380d1950 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 --2- 172.21.15.116:0/2316313365 >> v2:172.21.15.116:3300/0 conn(0x7ff5381622d0 0x7ff5380b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 -- 172.21.15.116:0/2316313365 >> 172.21.15.116:0/2316313365 conn(0x7ff53806c520 msgr2=0x7ff5380d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 -- 172.21.15.116:0/2316313365 shutdown_connections 2024-02-18T11:47:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f7138b52640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c005180 0x7f711c00b980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c490 tx=0x7f711c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63df9b6a4b2bd954 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c02ccd0 con 0x7f711c005180 2024-02-18T11:47:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.398+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1961051914 wait complete. 2024-02-18T11:47:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.399+0000 7f5ca150d640 1 -- start start 2024-02-18T11:47:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f5ca150d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f5ca150d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 0x7f5c9c0b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7fa1fcfec640 1 Processor -- start 2024-02-18T11:47:59.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c02ce50 con 0x7f711c005180 2024-02-18T11:47:59.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f78647d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78540e12a0 0x7f785404f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7fa1fcfec640 1 -- start start 2024-02-18T11:47:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7f5c9a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 0x7f5c9c0b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.400+0000 7ff53e72c640 1 -- 172.21.15.116:0/2316313365 wait complete. 2024-02-18T11:47:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f78647d0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854050870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5c9a7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 0x7f5c9c0b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57394/0 (socket says 172.21.15.116:57394) 2024-02-18T11:47:59.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5c9a7fc640 1 -- 172.21.15.116:0/1554488516 learned_addr learned my addr 172.21.15.116:0/1554488516 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f78647d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e2870 0x7f785400c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f786256c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78540e12a0 0x7f785404f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.428 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5ca150d640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c9c163350 0x7f5c9c163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f786256c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78540e12a0 0x7f785404f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57396/0 (socket says 172.21.15.116:57396) 2024-02-18T11:47:59.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c163d40 con 0x7f5c9c0b3f20 2024-02-18T11:47:59.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0b48e0 con 0x7f5c9c163350 2024-02-18T11:47:59.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7ff4c0a36640 1 Processor -- start 2024-02-18T11:47:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1581c0 con 0x7f5c9c0ccce0 2024-02-18T11:47:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f87056c9640 1 Processor -- start 2024-02-18T11:47:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f7132ffd640 1 -- 172.21.15.116:0/3104888105 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c034680 con 0x7f711c005180 2024-02-18T11:47:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5c9b7fe640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c9c163350 0x7f5c9c163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f87056c9640 1 -- start start 2024-02-18T11:47:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c005180 msgr2=0x7f711c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f7139353640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c005180 0x7f711c00b980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f712c00c490 tx=0x7f711c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5c9b7fe640 1 -- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 msgr2=0x7f5c9c157760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T11:47:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5c9b7fe640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c157760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f5c9b7fe640 1 -- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 msgr2=0x7f5c9c0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7ff4c0a36640 1 -- start start 2024-02-18T11:47:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f7861d6b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854050870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f78647d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7854050db0 con 0x7f78540e12a0 2024-02-18T11:47:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f78647d0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540503e0 con 0x7f78540e2870 2024-02-18T11:47:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f78647d0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785404f7d0 con 0x7f78540e4d20 2024-02-18T11:47:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.401+0000 7f786256c640 1 -- 172.21.15.116:0/2562013774 learned_addr learned my addr 172.21.15.116:0/2562013774 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 shutdown_connections 2024-02-18T11:47:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7139353640 1 --2- 172.21.15.116:0/3104888105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c005180 0x7f711c00b980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 >> 172.21.15.116:0/3104888105 conn(0x7f7134072650 msgr2=0x7f7134099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 shutdown_connections 2024-02-18T11:47:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7862d6d640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e2870 0x7f785400c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff53e72c640 1 Processor -- start 2024-02-18T11:47:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1fcfec640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 0x7fa1f80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1fcfec640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f81658b0 0x7fa1f8161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7861d6b640 1 -- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e2870 msgr2=0x7f785400c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7861d6b640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e2870 0x7f785400c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7861d6b640 1 -- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78540e12a0 msgr2=0x7f785404f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7861d6b640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78540e12a0 0x7f785404f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7861d6b640 1 -- 172.21.15.116:0/2562013774 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009400 con 0x7f78540e4d20 2024-02-18T11:47:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f434d95d640 1 --2- 172.21.15.116:0/2999097958 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340054b50 0x7f4340051af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f434804fcd0 tx=0x7f43480a6770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230fc9483690ae13 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff4c0a36640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40085d0 0x7ff4b40cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 0x7fa1f80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff53e72c640 1 -- start start 2024-02-18T11:47:59.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff4c0a36640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b40347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 0x7fa1f80cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56586/0 (socket says 172.21.15.116:56586) 2024-02-18T11:47:59.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff4c0a36640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff4b40ce400 0x7ff4b40d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f87056c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f803ad40 0x7f86f80e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1f77fe640 1 -- 172.21.15.116:0/1735050720 learned_addr learned my addr 172.21.15.116:0/1735050720 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff4c0a36640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4034ce0 con 0x7ff4b40085d0 2024-02-18T11:47:59.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1f8165cf0 0x7fa1f813cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff4c0a36640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40cfad0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f434d15c640 1 -- 172.21.15.116:0/2999097958 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480ac790 con 0x7f4340054b50 2024-02-18T11:47:59.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7fa1f6ffd640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f81658b0 0x7fa1f8161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7ff4c0a36640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b4033cc0 con 0x7ff4b40ce400 2024-02-18T11:47:59.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f87056c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f8050080 0x7f86f80366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.402+0000 7f7139353640 1 -- 172.21.15.116:0/3104888105 wait complete. 2024-02-18T11:47:59.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8204490 con 0x7fa1f8165cf0 2024-02-18T11:47:59.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8203810 con 0x7fa1f81658b0 2024-02-18T11:47:59.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f434d15c640 1 -- 172.21.15.116:0/2999097958 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43480ac930 con 0x7f4340054b50 2024-02-18T11:47:59.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8203af0 con 0x7fa1f8147ee0 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f87056c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f80df6e0 0x7f86f8036c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86feffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f803ad40 0x7f86f80e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86feffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f803ad40 0x7f86f80e10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:54152/0 (socket says 172.21.15.116:54152) 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86feffd640 1 -- 172.21.15.116:0/2391345844 learned_addr learned my addr 172.21.15.116:0/2391345844 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86fe7fc640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f8050080 0x7f86f80366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80cf020 con 0x7f86f80df6e0 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff4b40ce400 0x7ff4b40d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4baffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b40347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4bbfff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff4b40ce400 0x7ff4b40d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56590/0 (socket says 172.21.15.116:56590) 2024-02-18T11:47:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4bbfff640 1 -- 172.21.15.116:0/2914784085 learned_addr learned my addr 172.21.15.116:0/2914784085 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f434d15c640 1 -- 172.21.15.116:0/2999097958 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480acd80 con 0x7f4340054b50 2024-02-18T11:47:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80dd450 con 0x7f86f803ad40 2024-02-18T11:47:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f80e0800 con 0x7f86f8050080 2024-02-18T11:47:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f7861d6b640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854050870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f785c02d370 tx=0x7f785c068690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c04f060 con 0x7f78540e4d20 2024-02-18T11:47:59.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c067f20 con 0x7f78540e4d20 2024-02-18T11:47:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785404fab0 con 0x7f78540e4d20 2024-02-18T11:47:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c04f060 con 0x7f78540e4d20 2024-02-18T11:47:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f5c9b7fe640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 0x7f5c9c0b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f5c9b7fe640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900094d0 con 0x7f5c9c163350 2024-02-18T11:47:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86ff7fe640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f80df6e0 0x7f86f8036c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1f77fe640 1 -- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f81658b0 msgr2=0x7fa1f8161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1f77fe640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f81658b0 0x7fa1f8161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1f77fe640 1 -- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1f8165cf0 msgr2=0x7fa1f813cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1f77fe640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1f8165cf0 0x7fa1f813cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7fa1f77fe640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009b30 con 0x7fa1f8147ee0 2024-02-18T11:47:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f5c9a7fc640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 0x7f5c9c0b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f5c9b7fe640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c9c163350 0x7f5c9c163800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c94007d20 tx=0x7f5c9400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86fe7fc640 1 -- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f803ad40 msgr2=0x7f86f80e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86fe7fc640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f803ad40 0x7f86f80e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86fe7fc640 1 -- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f80df6e0 msgr2=0x7f86f8036c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86fe7fc640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f80df6e0 0x7f86f8036c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7f86fe7fc640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40094d0 con 0x7f86f8050080 2024-02-18T11:47:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 -- 172.21.15.116:0/2999097958 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340054b50 msgr2=0x7f4340051af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 --2- 172.21.15.116:0/2999097958 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340054b50 0x7f4340051af0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f434804fcd0 tx=0x7f43480a6770 comp rx=0 tx=0).stop 2024-02-18T11:47:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4baffd640 1 -- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff4b40ce400 msgr2=0x7ff4b40d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4baffd640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff4b40ce400 0x7ff4b40d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4baffd640 1 -- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40085d0 msgr2=0x7ff4b40cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4baffd640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40085d0 0x7ff4b40cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff53e72c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 0x7ff5380ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.403+0000 7ff4baffd640 1 -- 172.21.15.116:0/2914784085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4033fa0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f86fe7fc640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f8050080 0x7f86f80366f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8700050190 tx=0x7f8700068690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94018070 con 0x7f5c9c163350 2024-02-18T11:47:59.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff53e72c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5381622d0 0x7ff53815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff53e72c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff538162ca0 0x7ff538155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff4baffd640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b40347a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c400 tx=0x7ff4b000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400edd0 con 0x7f5c9c163350 2024-02-18T11:47:59.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c157ca0 con 0x7f5c9c163350 2024-02-18T11:47:59.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff537fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 0x7ff5380ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c164210 con 0x7f5c9c163350 2024-02-18T11:47:59.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94003bb0 con 0x7f5c9c163350 2024-02-18T11:47:59.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7854050070 con 0x7f78540e4d20 2024-02-18T11:47:59.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 -- 172.21.15.116:0/2999097958 shutdown_connections 2024-02-18T11:47:59.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 --2- 172.21.15.116:0/2999097958 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340054b50 0x7f4340051af0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 -- 172.21.15.116:0/2999097958 >> 172.21.15.116:0/2999097958 conn(0x7f4340020040 msgr2=0x7f4340056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f7139353640 1 Processor -- start 2024-02-18T11:47:59.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 -- 172.21.15.116:0/2999097958 shutdown_connections 2024-02-18T11:47:59.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f434f160640 1 -- 172.21.15.116:0/2999097958 wait complete. 2024-02-18T11:47:59.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7f7139353640 1 -- start start 2024-02-18T11:47:59.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7fa1f77fe640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 0x7fa1f80cdb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800e870 tx=0x7fa1e800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80094f0 con 0x7fa1f8147ee0 2024-02-18T11:47:59.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800c940 con 0x7fa1f8147ee0 2024-02-18T11:47:59.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800ccf0 con 0x7fa1f8147ee0 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00052d0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0005430 con 0x7ff4b40e42d0 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0003810 con 0x7ff4b40e42d0 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700073d80 con 0x7f86f8050080 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8700097400 con 0x7f86f8050080 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700097850 con 0x7f86f8050080 2024-02-18T11:47:59.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f804c4b0 con 0x7f86f8050080 2024-02-18T11:47:59.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f804ca70 con 0x7f86f8050080 2024-02-18T11:47:59.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff4c0a36640 1 -- 172.21.15.116:0/2914784085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b4034280 con 0x7ff4b40e42d0 2024-02-18T11:47:59.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f785c073b80 con 0x7f78540e4d20 2024-02-18T11:47:59.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff4c0a36640 1 -- 172.21.15.116:0/2914784085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b4033900 con 0x7ff4b40e42d0 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff537fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 0x7ff5380ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57428/0 (socket says 172.21.15.116:57428) 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff53e72c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380c7f40 con 0x7ff5380b5780 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff53e72c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380caf00 con 0x7ff5381622d0 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff53e72c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380cb1e0 con 0x7ff538162ca0 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff537fff640 1 -- 172.21.15.116:0/548321637 learned_addr learned my addr 172.21.15.116:0/548321637 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.404+0000 7ff5377fe640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5381622d0 0x7ff53815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff537fff640 1 -- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff538162ca0 msgr2=0x7ff538155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff537fff640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff538162ca0 0x7ff538155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff537fff640 1 -- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5381622d0 msgr2=0x7ff53815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff537fff640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5381622d0 0x7ff53815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.405+0000 7ff537fff640 1 -- 172.21.15.116:0/548321637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280094d0 con 0x7ff5380b5780 2024-02-18T11:47:59.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7ff537fff640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 0x7ff5380ca890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff53000c730 tx=0x7ff53000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f7852ffd640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f784006bcd0 0x7f784006e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f785c104750 con 0x7f78540e4d20 2024-02-18T11:47:59.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530003b70 con 0x7ff5380b5780 2024-02-18T11:47:59.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f434f160640 1 Processor -- start 2024-02-18T11:47:59.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f434f160640 1 -- start start 2024-02-18T11:47:59.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f7850ff9640 1 -- 172.21.15.116:0/2562013774 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78540057d0 con 0x7f78540e4d20 2024-02-18T11:47:59.456 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f786256c640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f784006bcd0 0x7f784006e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7139353640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 0x7f713415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7139353640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71340af6a0 0x7f71340c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7139353640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71341648f0 0x7f7134164d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7139353640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71341652b0 con 0x7f71341648f0 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7139353640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340cbe40 con 0x7f71340af6a0 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7139353640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340ca3b0 con 0x7f711c005180 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7133fff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 0x7f713415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f71337fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71340af6a0 0x7f71340c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7133fff640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 0x7f713415dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56624/0 (socket says 172.21.15.116:56624) 2024-02-18T11:47:59.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7138b52640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71341648f0 0x7f7134164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7133fff640 1 -- 172.21.15.116:0/959582699 learned_addr learned my addr 172.21.15.116:0/959582699 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f8203dd0 con 0x7fa1f8147ee0 2024-02-18T11:47:59.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f81612c0 con 0x7fa1f8147ee0 2024-02-18T11:47:59.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530003cd0 con 0x7ff5380b5780 2024-02-18T11:47:59.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40057e0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530003b70 con 0x7ff5380b5780 2024-02-18T11:47:59.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.406+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c005290 con 0x7f5c9c163350 2024-02-18T11:47:59.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5380cb4c0 con 0x7ff5380b5780 2024-02-18T11:47:59.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f786256c640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f784006bcd0 0x7f784006e190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f785802b910 tx=0x7f785803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f5c94014420 con 0x7f5c9c163350 2024-02-18T11:47:59.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f87000979b0 con 0x7f86f8050080 2024-02-18T11:47:59.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f80057d0 con 0x7f86f8050080 2024-02-18T11:47:59.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7133fff640 1 -- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71340af6a0 msgr2=0x7f71340c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7133fff640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71340af6a0 0x7f71340c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.407+0000 7f7133fff640 1 -- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71341648f0 msgr2=0x7f7134164d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f7133fff640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71341648f0 0x7f7134164d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f7133fff640 1 -- 172.21.15.116:0/959582699 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340c9df0 con 0x7f711c005180 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f7138b52640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71341648f0 0x7f7134164d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f86e77fe640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f86ec06bcc0 0x7f86ec06e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f7133fff640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 0x7f713415dc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f712400b7f0 tx=0x7f712400bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5381555d0 con 0x7ff5380b5780 2024-02-18T11:47:59.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f5c837fe640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5c8806bd70 0x7f5c8806e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5c9408bbc0 con 0x7f5c9c163350 2024-02-18T11:47:59.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f5c9affd640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5c8806bd70 0x7f5c8806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f87001093c0 con 0x7f86f8050080 2024-02-18T11:47:59.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f86feffd640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f86ec06bcc0 0x7f86ec06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f434f160640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 0x7f43400d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f434f160640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43400e12b0 0x7f43400e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f434f160640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434003be80 0x7f434003c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f8067510 con 0x7fa1f8147ee0 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f434e15e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 0x7f43400d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f434e15e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 0x7f43400d72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56640/0 (socket says 172.21.15.116:56640) 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.408+0000 7f434e15e640 1 -- 172.21.15.116:0/3397835101 learned_addr learned my addr 172.21.15.116:0/3397835101 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434004bec0 con 0x7f43400e12b0 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e95f640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434003be80 0x7f434003c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434004b2b0 con 0x7f434003be80 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434004b590 con 0x7f4340053c00 2024-02-18T11:47:59.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f5c9affd640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5c8806bd70 0x7f5c8806e230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c90036000 tx=0x7f5c90008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434d95d640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43400e12b0 0x7f43400e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e15e640 1 -- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434003be80 msgr2=0x7f434003c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e15e640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434003be80 0x7f434003c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e15e640 1 -- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43400e12b0 msgr2=0x7f43400e3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f86feffd640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f86ec06bcc0 0x7f86ec06e180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86f4034040 tx=0x7f86f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e15e640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43400e12b0 0x7f43400e3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e15e640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434004b870 con 0x7f4340053c00 2024-02-18T11:47:59.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434d95d640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43400e12b0 0x7f43400e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:47:59.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7f434e15e640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 0x7f43400d72c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f433800c480 tx=0x7f433800c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.409+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff538067840 con 0x7ff5380b5780 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.410+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff53000a850 con 0x7ff5380b5780 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.410+0000 7ff534ff9640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff52406bcc0 0x7ff52406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.410+0000 7ff5377fe640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff52406bcc0 0x7ff52406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.410+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff53000ce40 con 0x7ff5380b5780 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.410+0000 7ff5377fe640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff52406bcc0 0x7ff52406e180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.412+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c94056040 con 0x7f5c9c163350 2024-02-18T11:47:59.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.413+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f785c0d2730 con 0x7f78540e4d20 2024-02-18T11:47:59.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.414+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa1e8013470 con 0x7fa1f8147ee0 2024-02-18T11:47:59.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.414+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff4b0003970 con 0x7ff4b40e42d0 2024-02-18T11:47:59.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.414+0000 7fa1dffff640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa1e406bcc0 0x7fa1e406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.415+0000 7ff4a7fff640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff4a006bd40 0x7ff4a006e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.415+0000 7fa1f6ffd640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa1e406bcc0 0x7fa1e406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.415+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff4b008a510 con 0x7ff4b40e42d0 2024-02-18T11:47:59.469 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.415+0000 7ff4bb7fe640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff4a006bd40 0x7ff4a006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.415+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff530054f80 con 0x7ff5380b5780 2024-02-18T11:47:59.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.415+0000 7fa1f6ffd640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa1e406bcc0 0x7fa1e406e180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0035000 tx=0x7fa1f0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.416+0000 7ff4bb7fe640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff4a006bd40 0x7ff4a006e200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006a90 tx=0x7ff4a8006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.417+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4b0054a40 con 0x7ff4b40e42d0 2024-02-18T11:47:59.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.417+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338005ce0 con 0x7f4340053c00 2024-02-18T11:47:59.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.417+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4338005e40 con 0x7f4340053c00 2024-02-18T11:47:59.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.417+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa1e8059550 con 0x7fa1f8147ee0 2024-02-18T11:47:59.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87000d3940 con 0x7f86f8050080 2024-02-18T11:47:59.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.417+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434004bb50 con 0x7f4340053c00 2024-02-18T11:47:59.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43400e4040 con 0x7f4340053c00 2024-02-18T11:47:59.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380072f0 con 0x7f4340053c00 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124010830 con 0x7f711c005180 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124010990 con 0x7f711c005180 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340ca0d0 con 0x7f711c005180 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.418+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71340c9a70 con 0x7f711c005180 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.419+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124010d60 con 0x7f711c005180 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.419+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43400057e0 con 0x7f4340053c00 2024-02-18T11:47:59.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.419+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114005290 con 0x7f711c005180 2024-02-18T11:47:59.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.422+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1e80557f0 con 0x7fa1f8147ee0 2024-02-18T11:47:59.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.423+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4338007510 con 0x7f4340053c00 2024-02-18T11:47:59.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.424+0000 7f433effd640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f432c06bc70 0x7f432c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.424+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f433808a1c0 con 0x7f4340053c00 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.424+0000 7f434d95d640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f432c06bc70 0x7f432c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.425+0000 7f434d95d640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f432c06bc70 0x7f432c06e130 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f434000c190 tx=0x7f434806e030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.425+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7124027430 con 0x7f711c005180 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.425+0000 7f7130ff9640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f712006bd40 0x7f712006e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.425+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f712408b6b0 con 0x7f711c005180 2024-02-18T11:47:59.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.425+0000 7f71337fe640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f712006bd40 0x7f712006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:47:59.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.426+0000 7f71337fe640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f712006bd40 0x7f712006e200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71340b0a30 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:47:59.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.427+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4338054880 con 0x7f4340053c00 2024-02-18T11:47:59.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.432+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7124055be0 con 0x7f711c005180 2024-02-18T11:47:59.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.817+0000 7f5ca150d640 1 -- 172.21.15.116:0/1554488516 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5c7c005810 con 0x7f5c9c163350 2024-02-18T11:47:59.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.818+0000 7f5c837fe640 1 -- 172.21.15.116:0/1554488516 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c940098b0 con 0x7f5c9c163350 2024-02-18T11:47:59.812 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.821+0000 7f5c817fa640 1 -- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5c8806bd70 msgr2=0x7f5c8806e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.821+0000 7f5c817fa640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5c8806bd70 0x7f5c8806e230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c90036000 tx=0x7f5c90008030 comp rx=0 tx=0).stop 2024-02-18T11:47:59.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f7850ff9640 1 -- 172.21.15.116:0/2562013774 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7854054310 con 0x7f78540e4d20 2024-02-18T11:47:59.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 -- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c9c163350 msgr2=0x7f5c9c163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c9c163350 0x7f5c9c163800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c94007d20 tx=0x7f5c9400e4b0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 -- 172.21.15.116:0/1554488516 shutdown_connections 2024-02-18T11:47:59.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5c8806bd70 0x7f5c8806e230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c9c163350 0x7f5c9c163800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c9c0b3f20 0x7f5c9c0b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 --2- 172.21.15.116:0/1554488516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5c9c0ccce0 0x7f5c9c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 -- 172.21.15.116:0/1554488516 >> 172.21.15.116:0/1554488516 conn(0x7f5c9c072630 msgr2=0x7f5c9c0afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 -- 172.21.15.116:0/1554488516 shutdown_connections 2024-02-18T11:47:59.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f5c817fa640 1 -- 172.21.15.116:0/1554488516 wait complete. 2024-02-18T11:47:59.817 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f7852ffd640 1 -- 172.21.15.116:0/2562013774 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f785c0d22c0 con 0x7f78540e4d20 2024-02-18T11:47:59.817 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.822+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f86f8048ef0 con 0x7f86f8050080 2024-02-18T11:47:59.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.823+0000 7f86e77fe640 1 -- 172.21.15.116:0/2391345844 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f87000d7000 con 0x7f86f8050080 2024-02-18T11:47:59.818 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.824+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff5380ceda0 con 0x7ff5380b5780 2024-02-18T11:47:59.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.825+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f784006bcd0 msgr2=0x7f784006e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.825+0000 7f78647d0640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f784006bcd0 0x7f784006e190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f785802b910 tx=0x7f785803c040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 msgr2=0x7f7854050870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854050870 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f785c02d370 tx=0x7f785c068690 comp rx=0 tx=0).stop 2024-02-18T11:47:59.820 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.825+0000 7ff534ff9640 1 -- 172.21.15.116:0/548321637 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff5380ceda0 con 0x7ff5380b5780 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 shutdown_connections 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f784006bcd0 0x7f784006e190 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f785802b910 tx=0x7f785803c040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78540e2870 0x7f785400c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f78540e4d20 0x7f7854050870 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 --2- 172.21.15.116:0/2562013774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78540e12a0 0x7f785404f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.826+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 >> 172.21.15.116:0/2562013774 conn(0x7f7854020030 msgr2=0x7f78540092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.821 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f86ec06bcc0 msgr2=0x7f86ec06e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 shutdown_connections 2024-02-18T11:47:59.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7f78647d0640 1 -- 172.21.15.116:0/2562013774 wait complete. 2024-02-18T11:47:59.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff4b400f3c0 con 0x7ff4b40e42d0 2024-02-18T11:47:59.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7f87056c9640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f86ec06bcc0 0x7f86ec06e180 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f86f4034040 tx=0x7f86f4008030 comp rx=0 tx=0).stop 2024-02-18T11:47:59.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f8050080 msgr2=0x7f86f80366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7f87056c9640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f8050080 0x7f86f80366f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8700050190 tx=0x7f8700068690 comp rx=0 tx=0).stop 2024-02-18T11:47:59.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.827+0000 7ff4a7fff640 1 -- 172.21.15.116:0/2914784085 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4b0017020 con 0x7ff4b40e42d0 2024-02-18T11:47:59.823 INFO:teuthology.orchestra.run.smithi116.stdout:55834574857 2024-02-18T11:47:59.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff52406bcc0 msgr2=0x7ff52406e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7ff53e72c640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff52406bcc0 0x7ff52406e180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff52c006fd0 tx=0x7ff52c008040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 msgr2=0x7ff5380ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7ff53e72c640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 0x7ff5380ca890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff53000c730 tx=0x7ff53000cbf0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 shutdown_connections 2024-02-18T11:47:59.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f86ec06bcc0 0x7f86ec06e180 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f80df6e0 0x7f86f8036c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86f8050080 0x7f86f80366f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 --2- 172.21.15.116:0/2391345844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f803ad40 0x7f86f80e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 >> 172.21.15.116:0/2391345844 conn(0x7f86f8055340 msgr2=0x7f86f80dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 shutdown_connections 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.828+0000 7f87056c9640 1 -- 172.21.15.116:0/2391345844 wait complete. 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 shutdown_connections 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff52406bcc0 0x7ff52406e180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff538162ca0 0x7ff538155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5381622d0 0x7ff53815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.826 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 --2- 172.21.15.116:0/548321637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380b5780 0x7ff5380ca890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 >> 172.21.15.116:0/548321637 conn(0x7ff53806c520 msgr2=0x7ff5380cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 shutdown_connections 2024-02-18T11:47:59.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.830+0000 7ff53e72c640 1 -- 172.21.15.116:0/548321637 wait complete. 2024-02-18T11:47:59.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff4a006bd40 msgr2=0x7ff4a006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff4a006bd40 0x7ff4a006e200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006a90 tx=0x7ff4a8006a20 comp rx=0 tx=0).stop 2024-02-18T11:47:59.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 msgr2=0x7ff4b40347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b40347a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c400 tx=0x7ff4b000c8c0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 shutdown_connections 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff4a006bd40 0x7ff4a006e200 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff4b40ce400 0x7ff4b40d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4b40e42d0 0x7ff4b40347a0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000c400 tx=0x7ff4b000c8c0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 --2- 172.21.15.116:0/2914784085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40085d0 0x7ff4b40cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.831+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 >> 172.21.15.116:0/2914784085 conn(0x7ff4b401ef40 msgr2=0x7ff4b4007a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.832+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 shutdown_connections 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.832+0000 7ff4a5ffb640 1 -- 172.21.15.116:0/2914784085 wait complete. 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.832+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa1f8074520 con 0x7fa1f8147ee0 2024-02-18T11:47:59.829 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.833+0000 7fa1dffff640 1 -- 172.21.15.116:0/1735050720 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa1e8021090 con 0x7fa1f8147ee0 2024-02-18T11:47:59.830 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.830 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.837+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa1e406bcc0 msgr2=0x7fa1e406e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa1e406bcc0 0x7fa1e406e180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0035000 tx=0x7fa1f0030000 comp rx=0 tx=0).stop 2024-02-18T11:47:59.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 msgr2=0x7fa1f80cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 0x7fa1f80cdb30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800e870 tx=0x7fa1e800ed30 comp rx=0 tx=0).stop 2024-02-18T11:47:59.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 shutdown_connections 2024-02-18T11:47:59.831 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa1e406bcc0 0x7fa1e406e180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1f8165cf0 0x7fa1f813cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1f81658b0 0x7fa1f8161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 --2- 172.21.15.116:0/1735050720 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa1f8147ee0 0x7fa1f80cdb30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 >> 172.21.15.116:0/1735050720 conn(0x7fa1f80b12b0 msgr2=0x7fa1f80b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f43400e26a0 con 0x7f4340053c00 2024-02-18T11:47:59.832 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.838+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 shutdown_connections 2024-02-18T11:47:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.839+0000 7fa1fcfec640 1 -- 172.21.15.116:0/1735050720 wait complete. 2024-02-18T11:47:59.833 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.839+0000 7f433effd640 1 -- 172.21.15.116:0/3397835101 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f43400e26a0 con 0x7f4340053c00 2024-02-18T11:47:59.835 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-02-18T11:47:59.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f432c06bc70 msgr2=0x7f432c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f432c06bc70 0x7f432c06e130 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f434000c190 tx=0x7f434806e030 comp rx=0 tx=0).stop 2024-02-18T11:47:59.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 msgr2=0x7f43400d72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 0x7f43400d72c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f433800c480 tx=0x7f433800c940 comp rx=0 tx=0).stop 2024-02-18T11:47:59.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 shutdown_connections 2024-02-18T11:47:59.837 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f432c06bc70 0x7f432c06e130 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434003be80 0x7f434003c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43400e12b0 0x7f43400e3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 --2- 172.21.15.116:0/3397835101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4340053c00 0x7f43400d72c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 >> 172.21.15.116:0/3397835101 conn(0x7f4340020040 msgr2=0x7f43400d6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 shutdown_connections 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.843+0000 7f434f160640 1 -- 172.21.15.116:0/3397835101 wait complete. 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.844+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7114005810 con 0x7f711c005180 2024-02-18T11:47:59.838 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.844+0000 7f7130ff9640 1 -- 172.21.15.116:0/959582699 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71240592a0 con 0x7f711c005180 2024-02-18T11:47:59.839 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f712006bd40 msgr2=0x7f712006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f712006bd40 0x7f712006e200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f71340b0a30 tx=0x7f7128008040 comp rx=0 tx=0).stop 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 msgr2=0x7f713415dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 0x7f713415dc20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f712400b7f0 tx=0x7f712400bcb0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 shutdown_connections 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f712006bd40 0x7f712006e200 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71341648f0 0x7f7134164d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71340af6a0 0x7f71340c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 --2- 172.21.15.116:0/959582699 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f711c005180 0x7f713415dc20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:47:59.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.848+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 >> 172.21.15.116:0/959582699 conn(0x7f7134072650 msgr2=0x7f713415ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:47:59.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-02-18T11:47:59.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.849+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 shutdown_connections 2024-02-18T11:47:59.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:47:59.849+0000 7f7139353640 1 -- 172.21.15.116:0/959582699 wait complete. 2024-02-18T11:47:59.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-02-18T11:47:59.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-02-18T11:47:59.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-02-18T11:47:59.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-02-18T11:47:59.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-02-18T11:47:59.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-02-18T11:48:00.837 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-02-18T11:48:00.844 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-02-18T11:48:00.845 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-02-18T11:48:00.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-02-18T11:48:00.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-02-18T11:48:00.854 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-02-18T11:48:00.854 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-02-18T11:48:00.858 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-02-18T11:48:01.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.006+0000 7f3bf19cc640 1 Processor -- start 2024-02-18T11:48:01.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.006+0000 7f3bf19cc640 1 -- start start 2024-02-18T11:48:01.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf19cc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec147ee0 0x7f3bec071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf19cc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3bec1658b0 0x7f3bec0b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf19cc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3bec165cf0 0x7f3bec0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf19cc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0cb950 con 0x7f3bec165cf0 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf19cc640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec166290 con 0x7f3bec1658b0 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf19cc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec166570 con 0x7f3bec147ee0 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf11cb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3bec165cf0 0x7f3bec0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf11cb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3bec165cf0 0x7f3bec0b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57448/0 (socket says 172.21.15.116:57448) 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.008+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1209716454 learned_addr learned my addr 172.21.15.116:0/1209716454 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.009+0000 7f3bebfff640 1 --2- 172.21.15.116:0/1209716454 >> v2:172.21.15.143:3300/0 conn(0x7f3bec1658b0 0x7f3bec0b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.009+0000 7f3bf09ca640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec147ee0 0x7f3bec071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.009+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1209716454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec147ee0 msgr2=0x7f3bec071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.011+0000 7f3bf11cb640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec147ee0 0x7f3bec071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.011+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1209716454 >> v2:172.21.15.143:3300/0 conn(0x7f3bec1658b0 msgr2=0x7f3bec0b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.011+0000 7f3bf11cb640 1 --2- 172.21.15.116:0/1209716454 >> v2:172.21.15.143:3300/0 conn(0x7f3bec1658b0 0x7f3bec0b25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.011+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1209716454 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0cbc10 con 0x7f3bec165cf0 2024-02-18T11:48:01.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.012+0000 7f3bebfff640 1 --2- 172.21.15.116:0/1209716454 >> v2:172.21.15.143:3300/0 conn(0x7f3bec1658b0 0x7f3bec0b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:01.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.012+0000 7f3bf11cb640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0b2af0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3bec06f730 tx=0x7f3be402bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2629b34861b9a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.012+0000 7f3bf09ca640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec147ee0 0x7f3bec071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.012+0000 7f3beb7fe640 1 -- 172.21.15.116:0/1209716454 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be402cac0 con 0x7f3bec165cf0 2024-02-18T11:48:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.012+0000 7f3beb7fe640 1 -- 172.21.15.116:0/1209716454 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be402cc20 con 0x7f3bec165cf0 2024-02-18T11:48:01.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.012+0000 7f3beb7fe640 1 -- 172.21.15.116:0/1209716454 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be40343f0 con 0x7f3bec165cf0 2024-02-18T11:48:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1209716454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 msgr2=0x7f3bec0b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0b2af0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3bec06f730 tx=0x7f3be402bf20 comp rx=0 tx=0).stop 2024-02-18T11:48:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1209716454 shutdown_connections 2024-02-18T11:48:01.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0b2af0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 --2- 172.21.15.116:0/1209716454 >> v2:172.21.15.143:3300/0 conn(0x7f3bec1658b0 0x7f3bec0b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 --2- 172.21.15.116:0/1209716454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec147ee0 0x7f3bec071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1209716454 >> 172.21.15.116:0/1209716454 conn(0x7f3bec0b12b0 msgr2=0x7f3bec0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1209716454 shutdown_connections 2024-02-18T11:48:01.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.014+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1209716454 wait complete. 2024-02-18T11:48:01.009 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.016+0000 7f3bf19cc640 1 Processor -- start 2024-02-18T11:48:01.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.016+0000 7f3bf19cc640 1 -- start start 2024-02-18T11:48:01.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.018+0000 7f3bf19cc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec147ee0 0x7f3bec138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.018+0000 7f3bf19cc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec1658b0 0x7f3bec1669c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.018+0000 7f3bf19cc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf11cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf11cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57458/0 (socket says 172.21.15.116:57458) 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1026695715 learned_addr learned my addr 172.21.15.116:0/1026695715 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec139450 con 0x7f3bec165cf0 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0cd4a0 con 0x7f3bec147ee0 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0cd780 con 0x7f3bec1658b0 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.019+0000 7f3bf09ca640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec147ee0 0x7f3bec138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bebfff640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec1658b0 0x7f3bec1669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec1658b0 msgr2=0x7f3bec1669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bf11cb640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec1658b0 0x7f3bec1669c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec147ee0 msgr2=0x7f3bec138f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bf11cb640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec147ee0 0x7f3bec138f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bf11cb640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc009400 con 0x7f3bec165cf0 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.020+0000 7f3bf11cb640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0ce540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3be4036040 tx=0x7f3be4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.021+0000 7f3bf09ca640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec147ee0 0x7f3bec138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.021+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004040 con 0x7f3bec165cf0 2024-02-18T11:48:01.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.021+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be4009b30 con 0x7f3bec165cf0 2024-02-18T11:48:01.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.022+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0cdd40 con 0x7f3bec165cf0 2024-02-18T11:48:01.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.025+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be40041a0 con 0x7f3bec165cf0 2024-02-18T11:48:01.018 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.025+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4004510 con 0x7f3bec165cf0 2024-02-18T11:48:01.019 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.027+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3be40046b0 con 0x7f3bec165cf0 2024-02-18T11:48:01.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.027+0000 7f3be97fa640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3bd806bda0 0x7f3bd806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.028+0000 7f3bf09ca640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3bd806bda0 0x7f3bd806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.029+0000 7f3bf09ca640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3bd806bda0 0x7f3bd806e260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006fd0 tx=0x7f3be0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.029+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3be40ad530 con 0x7f3bec165cf0 2024-02-18T11:48:01.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.029+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc005290 con 0x7f3bec165cf0 2024-02-18T11:48:01.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.039+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3be4077a40 con 0x7f3bec165cf0 2024-02-18T11:48:01.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.061+0000 7f9a840eb640 1 Processor -- start 2024-02-18T11:48:01.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.061+0000 7f9a840eb640 1 -- start start 2024-02-18T11:48:01.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.060+0000 7f37aaa0e640 1 Processor -- start 2024-02-18T11:48:01.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.060+0000 7f37aaa0e640 1 -- start start 2024-02-18T11:48:01.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f9a840eb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a7c1621f0 0x7f9a7c099a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f9a840eb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a7c0c4de0 0x7f9a7c0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f9a840eb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c099f90 con 0x7f9a7c06bc70 2024-02-18T11:48:01.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f9a840eb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c09a230 con 0x7f9a7c1621f0 2024-02-18T11:48:01.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f9a840eb640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c09a510 con 0x7f9a7c0c4de0 2024-02-18T11:48:01.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a7c0c4de0 0x7f9a7c0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a7c0c4de0 0x7f9a7c0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:54202/0 (socket says 172.21.15.116:54202) 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37aaa0e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c100cf0 0x7f379c03a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37aaa0e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f379c104970 0x7f379c104dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37aaa0e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379c105310 con 0x7f379c005ec0 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37aaa0e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379c037b50 con 0x7f379c104970 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37aaa0e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c007340 con 0x7f379c100cf0 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37a8fab640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f379c005ec0 0x7f379c006350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:45998/0 (socket says 172.21.15.116:45998) 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37a8fab640 1 -- 172.21.15.116:0/892185458 learned_addr learned my addr 172.21.15.116:0/892185458 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:01.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37a37fe640 1 --2- 172.21.15.116:0/892185458 >> v2:172.21.15.143:3300/0 conn(0x7f379c104970 0x7f379c104dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 -- 172.21.15.116:0/2377593322 learned_addr learned my addr 172.21.15.116:0/2377593322 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7f37a3fff640 1 --2- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c100cf0 0x7f379c03a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f37a37fe640 1 -- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c100cf0 msgr2=0x7f379c03a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f37a37fe640 1 --2- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c100cf0 0x7f379c03a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f37a37fe640 1 -- 172.21.15.116:0/892185458 >> v1:172.21.15.116:6789/0 conn(0x7f379c005ec0 legacy=0x7f379c006350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f37a37fe640 1 -- 172.21.15.116:0/892185458 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c007620 con 0x7f379c104970 2024-02-18T11:48:01.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 -- 172.21.15.116:0/2377593322 >> v2:172.21.15.116:3300/0 conn(0x7f9a7c1621f0 msgr2=0x7f9a7c099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7fc12121f640 1 Processor -- start 2024-02-18T11:48:01.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.062+0000 7fc12121f640 1 -- start start 2024-02-18T11:48:01.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 --2- 172.21.15.116:0/2377593322 >> v2:172.21.15.116:3300/0 conn(0x7f9a7c1621f0 0x7f9a7c099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7fc12121f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c16a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7fc12121f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc11c0b25b0 0x7fc11c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7fc12121f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0716d0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7fc12121f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c0cc360 con 0x7fc11c0b25b0 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7fc12121f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0cc640 con 0x7fc11c0cbe10 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.064+0000 7fc120a1e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc11c0b25b0 0x7fc11c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.064+0000 7fc11bfff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc11c16a2a0 0x7fc11c16ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46006/0 (socket says 172.21.15.116:46006) 2024-02-18T11:48:01.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.064+0000 7fc11bfff640 1 -- 172.21.15.116:0/4277213045 learned_addr learned my addr 172.21.15.116:0/4277213045 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:01.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.064+0000 7f7f4719a640 1 Processor -- start 2024-02-18T11:48:01.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.064+0000 7f7f4719a640 1 -- start start 2024-02-18T11:48:01.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7fc11affd640 1 -- 172.21.15.116:0/4277213045 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206076296 0 0) 0x7fc11c0716d0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7fc11affd640 1 -- 172.21.15.116:0/4277213045 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc104005730 con 0x7fc11c16a2a0 2024-02-18T11:48:01.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7fc11b7fe640 1 --2- 172.21.15.116:0/4277213045 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c16a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f4719a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f38100d90 0x7f7f38037c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f4719a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f3804f280 0x7f7f3804f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f4719a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f38052ce0 con 0x7f7f38053d10 2024-02-18T11:48:01.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f4719a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f38052f50 con 0x7f7f38100d90 2024-02-18T11:48:01.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f4719a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f38053230 con 0x7f7f3804f280 2024-02-18T11:48:01.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f44f36640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f38100d90 0x7f7f38037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f44f36640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f38100d90 0x7f7f38037c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57472/0 (socket says 172.21.15.116:57472) 2024-02-18T11:48:01.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f44f36640 1 -- 172.21.15.116:0/178602535 learned_addr learned my addr 172.21.15.116:0/178602535 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 -- 172.21.15.116:0/2377593322 >> v1:172.21.15.143:6789/0 conn(0x7f9a7c06bc70 legacy=0x7f9a7c0c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.063+0000 7f9a82688640 1 -- 172.21.15.116:0/2377593322 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c09a7f0 con 0x7f9a7c0c4de0 2024-02-18T11:48:01.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.064+0000 7f9a82688640 1 --2- 172.21.15.116:0/2377593322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0b0670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c166c20 tx=0x7f9a7402bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc9128fec9f6bc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7fc120a1e640 1 -- 172.21.15.116:0/4277213045 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 msgr2=0x7fc11c16a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7fc120a1e640 1 --2- 172.21.15.116:0/4277213045 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c16a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7fc120a1e640 1 -- 172.21.15.116:0/4277213045 >> v1:172.21.15.116:6789/0 conn(0x7fc11c16a2a0 legacy=0x7fc11c16ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7fc120a1e640 1 -- 172.21.15.116:0/4277213045 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0cc920 con 0x7fc11c0b25b0 2024-02-18T11:48:01.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f37a37fe640 1 --2- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c104970 0x7f379c104dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3790008820 tx=0x7f3790034270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0ce425087aecf server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f9a80e85640 1 -- 172.21.15.116:0/2377593322 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7402caa0 con 0x7f9a7c0c4de0 2024-02-18T11:48:01.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7fa349725640 1 Processor -- start 2024-02-18T11:48:01.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f9a80e85640 1 -- 172.21.15.116:0/2377593322 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7402cc00 con 0x7f9a7c0c4de0 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f9a840eb640 1 -- 172.21.15.116:0/2377593322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c0c4de0 msgr2=0x7f9a7c0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f9a840eb640 1 --2- 172.21.15.116:0/2377593322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0b0670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c166c20 tx=0x7f9a7402bf00 comp rx=0 tx=0).stop 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f9a840eb640 1 -- 172.21.15.116:0/2377593322 shutdown_connections 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f9a840eb640 1 --2- 172.21.15.116:0/2377593322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0b0670 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f9a840eb640 1 --2- 172.21.15.116:0/2377593322 >> v2:172.21.15.116:3300/0 conn(0x7f9a7c1621f0 0x7f9a7c099a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f9a840eb640 1 -- 172.21.15.116:0/2377593322 >> 172.21.15.116:0/2377593322 conn(0x7f9a7c0b31f0 msgr2=0x7f9a7c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.065+0000 7f7f45737640 1 --2- 172.21.15.116:0/178602535 >> v2:172.21.15.143:3300/0 conn(0x7f7f3804f280 0x7f7f3804f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f7f44f36640 1 -- 172.21.15.116:0/178602535 >> v2:172.21.15.143:3300/0 conn(0x7f7f3804f280 msgr2=0x7f7f3804f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f7f44f36640 1 --2- 172.21.15.116:0/178602535 >> v2:172.21.15.143:3300/0 conn(0x7f7f3804f280 0x7f7f3804f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f7f44f36640 1 -- 172.21.15.116:0/178602535 >> v1:172.21.15.116:6789/0 conn(0x7f7f38053d10 legacy=0x7f7f38054130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f7f3f7fe640 1 -- 172.21.15.116:0/178602535 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609531326 0 0) 0x7f7f38052ce0 con 0x7f7f38053d10 2024-02-18T11:48:01.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7f7f44f36640 1 -- 172.21.15.116:0/178602535 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38053510 con 0x7f7f38100d90 2024-02-18T11:48:01.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fefbeda0640 1 Processor -- start 2024-02-18T11:48:01.071 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.066+0000 7fa349725640 1 -- start start 2024-02-18T11:48:01.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fc120a1e640 1 --2- 172.21.15.116:0/4277213045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c0b2a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc114007d60 tx=0x7fc11402bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb1503e1b518eb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.072 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fefbeda0640 1 -- start start 2024-02-18T11:48:01.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7f37a2ffd640 1 -- 172.21.15.116:0/892185458 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379003c5f0 con 0x7f379c104970 2024-02-18T11:48:01.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7f37a2ffd640 1 -- 172.21.15.116:0/892185458 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379003c750 con 0x7f379c104970 2024-02-18T11:48:01.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7f37a2ffd640 1 -- 172.21.15.116:0/892185458 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379003cae0 con 0x7f379c104970 2024-02-18T11:48:01.073 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fa349725640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 0x7fa33c0df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fa349725640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa33c04fee0 0x7fa33c054260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fa349725640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c006890 con 0x7fa33c04fa80 2024-02-18T11:48:01.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fa349725640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c007760 con 0x7fa33c04fee0 2024-02-18T11:48:01.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7fa349725640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c006dc0 con 0x7fa33c00aa90 2024-02-18T11:48:01.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 -- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c104970 msgr2=0x7f379c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c104970 0x7f379c104dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3790008820 tx=0x7f3790034270 comp rx=0 tx=0).stop 2024-02-18T11:48:01.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa342ffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 0x7fa33c0df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa3437fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa33c04fee0 0x7fa33c054260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa3437fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa33c04fee0 0x7fa33c054260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57482/0 (socket says 172.21.15.116:57482) 2024-02-18T11:48:01.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa342ffd640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 0x7fa33c0df1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56688/0 (socket says 172.21.15.116:56688) 2024-02-18T11:48:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa3437fe640 1 -- 172.21.15.116:0/605982342 learned_addr learned my addr 172.21.15.116:0/605982342 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fefbeda0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb00df410 0x7fefb00382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 -- 172.21.15.116:0/892185458 shutdown_connections 2024-02-18T11:48:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c104970 0x7f379c104dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/892185458 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c100cf0 0x7f379c03a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 -- 172.21.15.116:0/892185458 >> 172.21.15.116:0/892185458 conn(0x7f379c009fe0 msgr2=0x7f379c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 -- 172.21.15.116:0/892185458 shutdown_connections 2024-02-18T11:48:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f37aaa0e640 1 -- 172.21.15.116:0/892185458 wait complete. 2024-02-18T11:48:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fefbeda0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fefb0054260 0x7fefb0038f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa3437fe640 1 -- 172.21.15.116:0/605982342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 msgr2=0x7fa33c0df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fefbeda0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb0006890 con 0x7fefb000aa90 2024-02-18T11:48:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fefbeda0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb0007760 con 0x7fefb0054260 2024-02-18T11:48:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fefbeda0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0006ae0 con 0x7fefb00df410 2024-02-18T11:48:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7fa3437fe640 1 --2- 172.21.15.116:0/605982342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 0x7fa33c0df1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7f7f44f36640 1 --2- 172.21.15.116:0/178602535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f38100d90 0x7f7f38037c30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002d370 tx=0x7f7f400958f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a568e58dfdccbbe8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fc11affd640 1 -- 172.21.15.116:0/4277213045 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c001670 con 0x7fc11c0b25b0 2024-02-18T11:48:01.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fa3437fe640 1 -- 172.21.15.116:0/605982342 >> v1:172.21.15.116:6789/0 conn(0x7fa33c04fa80 legacy=0x7fa33c0382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fa3437fe640 1 -- 172.21.15.116:0/605982342 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0070a0 con 0x7fa33c04fee0 2024-02-18T11:48:01.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fc11affd640 1 -- 172.21.15.116:0/4277213045 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11402c7b0 con 0x7fc11c0b25b0 2024-02-18T11:48:01.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fc11affd640 1 -- 172.21.15.116:0/4277213045 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11402cb40 con 0x7fc11c0b25b0 2024-02-18T11:48:01.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7f7f3f7fe640 1 -- 172.21.15.116:0/178602535 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4009b700 con 0x7f7f38100d90 2024-02-18T11:48:01.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7f7f3f7fe640 1 -- 172.21.15.116:0/178602535 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4009b8a0 con 0x7f7f38100d90 2024-02-18T11:48:01.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7f7f3f7fe640 1 -- 172.21.15.116:0/178602535 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4009bcd0 con 0x7f7f38100d90 2024-02-18T11:48:01.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 -- 172.21.15.116:0/4277213045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 msgr2=0x7fc11c0b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 --2- 172.21.15.116:0/4277213045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c0b2a40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc114007d60 tx=0x7fc11402bc80 comp rx=0 tx=0).stop 2024-02-18T11:48:01.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fa342ffd640 1 --2- 172.21.15.116:0/605982342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 0x7fa33c0df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:01.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 -- 172.21.15.116:0/4277213045 shutdown_connections 2024-02-18T11:48:01.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 --2- 172.21.15.116:0/4277213045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c0b2a40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 --2- 172.21.15.116:0/4277213045 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c16a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 -- 172.21.15.116:0/4277213045 >> 172.21.15.116:0/4277213045 conn(0x7fc11c0b12b0 msgr2=0x7fc11c0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f37aaa0e640 1 Processor -- start 2024-02-18T11:48:01.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 -- 172.21.15.116:0/178602535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f38100d90 msgr2=0x7f7f38037c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 -- 172.21.15.116:0/4277213045 shutdown_connections 2024-02-18T11:48:01.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f37aaa0e640 1 -- start start 2024-02-18T11:48:01.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fa3437fe640 1 --2- 172.21.15.116:0/605982342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c054260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa334009870 tx=0x7fa33402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b3731d65c8696c server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7fc12121f640 1 -- 172.21.15.116:0/4277213045 wait complete. 2024-02-18T11:48:01.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 --2- 172.21.15.116:0/178602535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f38100d90 0x7f7f38037c30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002d370 tx=0x7f7f400958f0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7fa341ffb640 1 -- 172.21.15.116:0/605982342 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33402f850 con 0x7fa33c04fee0 2024-02-18T11:48:01.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7fa341ffb640 1 -- 172.21.15.116:0/605982342 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33402f9b0 con 0x7fa33c04fee0 2024-02-18T11:48:01.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7fa341ffb640 1 -- 172.21.15.116:0/605982342 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33402f850 con 0x7fa33c04fee0 2024-02-18T11:48:01.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 -- 172.21.15.116:0/178602535 shutdown_connections 2024-02-18T11:48:01.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fefbdd9e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb00df410 0x7fefb00382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fefbe59f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fefb0054260 0x7fefb0038f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fefbd59d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fefb000aa90 0x7fefb00387f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46038/0 (socket says 172.21.15.116:46038) 2024-02-18T11:48:01.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7fefbd59d640 1 -- 172.21.15.116:0/1051593670 learned_addr learned my addr 172.21.15.116:0/1051593670 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:01.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7f9a840eb640 1 -- 172.21.15.116:0/2377593322 shutdown_connections 2024-02-18T11:48:01.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.067+0000 7f9a840eb640 1 -- 172.21.15.116:0/2377593322 wait complete. 2024-02-18T11:48:01.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.068+0000 7f9a840eb640 1 Processor -- start 2024-02-18T11:48:01.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 --2- 172.21.15.116:0/178602535 >> v2:172.21.15.143:3300/0 conn(0x7f7f3804f280 0x7f7f3804f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 --2- 172.21.15.116:0/178602535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f38100d90 0x7f7f38037c30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.069+0000 7f9a840eb640 1 -- start start 2024-02-18T11:48:01.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 -- 172.21.15.116:0/178602535 >> 172.21.15.116:0/178602535 conn(0x7f7f38009fa0 msgr2=0x7f7f3800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f7f4719a640 1 -- 172.21.15.116:0/178602535 shutdown_connections 2024-02-18T11:48:01.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f7f4719a640 1 -- 172.21.15.116:0/178602535 wait complete. 2024-02-18T11:48:01.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7fefbe59f640 1 -- 172.21.15.116:0/1051593670 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb00df410 msgr2=0x7fefb00382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7fefbe59f640 1 --2- 172.21.15.116:0/1051593670 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb00df410 0x7fefb00382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7fefbe59f640 1 -- 172.21.15.116:0/1051593670 >> v1:172.21.15.116:6789/0 conn(0x7fefb000aa90 legacy=0x7fefb00387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a840eb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c06bc70 0x7f9a7c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a840eb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a840eb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a7c1621f0 0x7f9a7c08a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a840eb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c162900 con 0x7f9a7c0c4de0 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a840eb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0cb6f0 con 0x7f9a7c06bc70 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a840eb640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c1653e0 con 0x7f9a7c1621f0 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a82688640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a7c1621f0 0x7f9a7c08a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a82688640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a7c1621f0 0x7f9a7c08a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56708/0 (socket says 172.21.15.116:56708) 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a82688640 1 -- 172.21.15.116:0/1265778840 learned_addr learned my addr 172.21.15.116:0/1265778840 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.070+0000 7f9a81e87640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c06bc70 0x7f9a7c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81686640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81e87640 1 -- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a7c1621f0 msgr2=0x7f9a7c08a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81e87640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a7c1621f0 0x7f9a7c08a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81e87640 1 -- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a7c0c4de0 msgr2=0x7f9a7c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81e87640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81e87640 1 -- 172.21.15.116:0/1265778840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74009b30 con 0x7f9a7c06bc70 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81686640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a81e87640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c06bc70 0x7f9a7c0c87d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b470 tx=0x7f9a7800b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78011830 con 0x7f9a7c06bc70 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37aaa0e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c005ec0 0x7f379c0fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37aaa0e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 0x7f379c02b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37aaa0e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c104970 0x7f379c04bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37aaa0e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c102c70 con 0x7f379c100cf0 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37aaa0e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c0fe630 con 0x7f379c005ec0 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c005ec0 0x7f379c0fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37aaa0e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379c04d070 con 0x7f379c104970 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37a37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 0x7f379c02b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37a37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 0x7f379c02b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57508/0 (socket says 172.21.15.116:57508) 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f37a37fe640 1 -- 172.21.15.116:0/3541985196 learned_addr learned my addr 172.21.15.116:0/3541985196 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a8fab640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c104970 0x7f379c04bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a37fe640 1 -- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c104970 msgr2=0x7f379c04bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a37fe640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c104970 0x7f379c04bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a37fe640 1 -- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c005ec0 msgr2=0x7f379c0fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.091 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a37fe640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c005ec0 0x7f379c0fe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a37fe640 1 -- 172.21.15.116:0/3541985196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3790008480 con 0x7f379c100cf0 2024-02-18T11:48:01.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f37a37fe640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 0x7f379c02b4a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f379003e040 tx=0x7f37900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a780119b0 con 0x7f9a7c06bc70 2024-02-18T11:48:01.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.071+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78011d60 con 0x7f9a7c06bc70 2024-02-18T11:48:01.092 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7fefbe59f640 1 -- 172.21.15.116:0/1051593670 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb0006dc0 con 0x7fefb0054260 2024-02-18T11:48:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7c164ab0 con 0x7f9a7c06bc70 2024-02-18T11:48:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c165070 con 0x7f9a7c06bc70 2024-02-18T11:48:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7f7f4719a640 1 Processor -- start 2024-02-18T11:48:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7fc12121f640 1 Processor -- start 2024-02-18T11:48:01.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.072+0000 7fc12121f640 1 -- start start 2024-02-18T11:48:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900185a0 con 0x7f379c100cf0 2024-02-18T11:48:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c04c510 con 0x7f379c100cf0 2024-02-18T11:48:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379003b670 con 0x7f379c100cf0 2024-02-18T11:48:01.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7f7f4719a640 1 -- start start 2024-02-18T11:48:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7fa349725640 1 -- 172.21.15.116:0/605982342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 msgr2=0x7fa33c054260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7fa349725640 1 --2- 172.21.15.116:0/605982342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c054260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa334009870 tx=0x7fa33402ba30 comp rx=0 tx=0).stop 2024-02-18T11:48:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c04cab0 con 0x7f379c100cf0 2024-02-18T11:48:01.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f4719a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 0x7f7f38194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f4719a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f38053d10 0x7f7f380fed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379003b9e0 con 0x7f379c100cf0 2024-02-18T11:48:01.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7c067840 con 0x7f9a7c06bc70 2024-02-18T11:48:01.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f4719a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38100d90 0x7f7f3804e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f4719a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f381935b0 con 0x7f7f3804f280 2024-02-18T11:48:01.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f4719a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38193890 con 0x7f7f38100d90 2024-02-18T11:48:01.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f4719a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38193b70 con 0x7f7f38053d10 2024-02-18T11:48:01.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.073+0000 7fc12121f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c16a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7fc12121f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7fc12121f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c16a2a0 0x7fc11c162670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f44f36640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 0x7f7f38194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f44f36640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 0x7f7f38194650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57528/0 (socket says 172.21.15.116:57528) 2024-02-18T11:48:01.098 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f7f44f36640 1 -- 172.21.15.116:0/436104119 learned_addr learned my addr 172.21.15.116:0/436104119 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7fc12121f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c16abc0 con 0x7fc11c0b25b0 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c16a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c16a2a0 0x7fc11c162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc11bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c16a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57522/0 (socket says 172.21.15.116:57522) 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc11bfff640 1 -- 172.21.15.116:0/3500278780 learned_addr learned my addr 172.21.15.116:0/3500278780 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c16ae60 con 0x7fc11c16a2a0 2024-02-18T11:48:01.099 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c16b140 con 0x7fc11c0cbe10 2024-02-18T11:48:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f3ffff640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f38053d10 0x7f7f380fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3784005290 con 0x7f379c100cf0 2024-02-18T11:48:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f45737640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38100d90 0x7f7f3804e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc11b7fe640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fa349725640 1 -- 172.21.15.116:0/605982342 shutdown_connections 2024-02-18T11:48:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fa349725640 1 --2- 172.21.15.116:0/605982342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c054260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fa349725640 1 --2- 172.21.15.116:0/605982342 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c00aa90 0x7fa33c0df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fa349725640 1 -- 172.21.15.116:0/605982342 >> 172.21.15.116:0/605982342 conn(0x7fa33c009fa0 msgr2=0x7fa33c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.101 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f44f36640 1 -- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f38053d10 msgr2=0x7f7f380fed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f44f36640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f38053d10 0x7f7f380fed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f44f36640 1 -- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38100d90 msgr2=0x7f7f3804e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f44f36640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38100d90 0x7f7f3804e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f44f36640 1 -- 172.21.15.116:0/436104119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4004fe10 con 0x7f7f3804f280 2024-02-18T11:48:01.102 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9a7801b460 con 0x7f9a7c06bc70 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 -- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 msgr2=0x7fc11c162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 -- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 msgr2=0x7fc11c16a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c16a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140079c0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f7f44f36640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 0x7f7f38194650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f4004f440 tx=0x7f7f400741f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.103 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.074+0000 7fefbe59f640 1 --2- 172.21.15.116:0/1051593670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb0054260 0x7fefb0038f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fefb8000b60 tx=0x7fefb80a6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2aaf875029d6bc server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fefbcd9c640 1 -- 172.21.15.116:0/1051593670 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb80ac7b0 con 0x7fefb0054260 2024-02-18T11:48:01.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fefbcd9c640 1 -- 172.21.15.116:0/1051593670 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb80ac910 con 0x7fefb0054260 2024-02-18T11:48:01.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fefbcd9c640 1 -- 172.21.15.116:0/1051593670 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb80acd40 con 0x7fefb0054260 2024-02-18T11:48:01.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7fc120a1e640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c16a2a0 0x7fc11c162670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc114033000 tx=0x7fc114004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.104 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.075+0000 7f9ebfe81640 1 Processor -- start 2024-02-18T11:48:01.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f9a727fc640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a6006bd40 0x7f9a6006e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f9ebfe81640 1 -- start start 2024-02-18T11:48:01.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f400a4bc0 con 0x7f7f3804f280 2024-02-18T11:48:01.105 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9a7808a700 con 0x7f9a7c06bc70 2024-02-18T11:48:01.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f400a9070 con 0x7f7f3804f280 2024-02-18T11:48:01.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7fa349725640 1 -- 172.21.15.116:0/605982342 shutdown_connections 2024-02-18T11:48:01.106 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f38193e50 con 0x7f7f3804f280 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f380ff280 con 0x7f7f3804f280 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7fefbeda0640 1 -- 172.21.15.116:0/1051593670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb0054260 msgr2=0x7fefb0038f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7fefbeda0640 1 --2- 172.21.15.116:0/1051593670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb0054260 0x7fefb0038f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fefb8000b60 tx=0x7fefb80a6790 comp rx=0 tx=0).stop 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7fefbeda0640 1 -- 172.21.15.116:0/1051593670 shutdown_connections 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7fefbeda0640 1 --2- 172.21.15.116:0/1051593670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb0054260 0x7fefb0038f00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7fefbeda0640 1 --2- 172.21.15.116:0/1051593670 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb00df410 0x7fefb00382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7fefbeda0640 1 -- 172.21.15.116:0/1051593670 >> 172.21.15.116:0/1051593670 conn(0x7fefb0009fa0 msgr2=0x7fefb000a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7f9a81686640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a6006bd40 0x7f9a6006e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9a81686640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a6006bd40 0x7f9a6006e200 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0d15b0 tx=0x7f9a6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f400a5430 con 0x7f7f3804f280 2024-02-18T11:48:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebfe81640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9eb00e3290 0x7f9eb004f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7fefbeda0640 1 -- 172.21.15.116:0/1051593670 shutdown_connections 2024-02-18T11:48:01.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7fefbeda0640 1 -- 172.21.15.116:0/1051593670 wait complete. 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebfe81640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb0038090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebfe81640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb00e3670 con 0x7f9eb004fee0 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebfe81640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb00068d0 con 0x7f9eb00e3290 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebfe81640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0006bb0 con 0x7f9eb0054490 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebd41c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9eb004fee0 0x7f9eb0053d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46044/0 (socket says 172.21.15.116:46044) 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebd41c640 1 -- 172.21.15.116:0/2269681371 learned_addr learned my addr 172.21.15.116:0/2269681371 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebdc1d640 1 --2- 172.21.15.116:0/2269681371 >> v2:172.21.15.143:3300/0 conn(0x7f9eb00e3290 0x7f9eb004f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.076+0000 7fa349725640 1 -- 172.21.15.116:0/605982342 wait complete. 2024-02-18T11:48:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebe41e640 1 --2- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb0038090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f379003bb40 con 0x7f379c100cf0 2024-02-18T11:48:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 msgr2=0x7f9eb0038090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebdc1d640 1 --2- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb0038090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.077+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2269681371 >> v1:172.21.15.116:6789/0 conn(0x7f9eb004fee0 legacy=0x7f9eb0053d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.078+0000 7f37a0ff9640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f378806bca0 0x7f378806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.078+0000 7fa349725640 1 Processor -- start 2024-02-18T11:48:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.078+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f37900ad200 con 0x7f379c100cf0 2024-02-18T11:48:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.078+0000 7fa349725640 1 -- start start 2024-02-18T11:48:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.078+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2269681371 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0006e90 con 0x7f9eb00e3290 2024-02-18T11:48:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.078+0000 7f37a3fff640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f378806bca0 0x7f378806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fefbeda0640 1 Processor -- start 2024-02-18T11:48:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f20005290 con 0x7f7f3804f280 2024-02-18T11:48:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7f9ebdc1d640 1 --2- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb00e3290 0x7f9eb004f9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097a0 tx=0x7f9eb402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d36828c56920eeb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fefbeda0640 1 -- start start 2024-02-18T11:48:01.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7f37a3fff640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f378806bca0 0x7f378806e160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f379800bac0 tx=0x7f3798008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa349725640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa33c00aa90 0x7fa33c0d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa349725640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c04fa80 0x7fa33c0e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa349725640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c0da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa349725640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0e3b80 con 0x7fa33c04fee0 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa349725640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0e38a0 con 0x7fa33c00aa90 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa349725640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c049880 con 0x7fa33c04fa80 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa342ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa33c00aa90 0x7fa33c0d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa3437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c0da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa342ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa33c00aa90 0x7fa33c0d8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:54260/0 (socket says 172.21.15.116:54260) 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa342ffd640 1 -- 172.21.15.116:0/2181425689 learned_addr learned my addr 172.21.15.116:0/2181425689 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.079+0000 7fa3427fc640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c04fa80 0x7fa33c0e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fa3437fe640 1 -- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c04fa80 msgr2=0x7fa33c0e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fa3437fe640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c04fa80 0x7fa33c0e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fa3437fe640 1 -- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa33c00aa90 msgr2=0x7fa33c0d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fa3437fe640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa33c00aa90 0x7fa33c0d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fa3437fe640 1 -- 172.21.15.116:0/2181425689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34404fe10 con 0x7fa33c04fee0 2024-02-18T11:48:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbeda0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefb000aa90 0x7fefb004a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbeda0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb0054260 0x7fefb004a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbeda0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 0x7fefb00e17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbeda0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb00d6b60 con 0x7fefb00df410 2024-02-18T11:48:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbeda0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb00d5360 con 0x7fefb000aa90 2024-02-18T11:48:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbeda0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb004ad00 con 0x7fefb0054260 2024-02-18T11:48:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fa3437fe640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c0da8c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa334033000 tx=0x7fa334004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbe59f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 0x7fefb00e17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbe59f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 0x7fefb00e17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57538/0 (socket says 172.21.15.116:57538) 2024-02-18T11:48:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbe59f640 1 -- 172.21.15.116:0/3230948459 learned_addr learned my addr 172.21.15.116:0/3230948459 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbd59d640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb0054260 0x7fefb004a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114034c00 con 0x7fc11c16a2a0 2024-02-18T11:48:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c16b420 con 0x7fc11c16a2a0 2024-02-18T11:48:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114034da0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fefbdd9e640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefb000aa90 0x7fefb004a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114040690 con 0x7fc11c16a2a0 2024-02-18T11:48:01.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c16b9e0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.080+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a78054b80 con 0x7f9a7c06bc70 2024-02-18T11:48:01.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebcc1b640 1 -- 172.21.15.116:0/2269681371 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb402d040 con 0x7f9eb00e3290 2024-02-18T11:48:01.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebcc1b640 1 -- 172.21.15.116:0/2269681371 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb4031810 con 0x7f9eb00e3290 2024-02-18T11:48:01.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebcc1b640 1 -- 172.21.15.116:0/2269681371 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4031ba0 con 0x7f9eb00e3290 2024-02-18T11:48:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7fefbe59f640 1 -- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb0054260 msgr2=0x7fefb004a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7fefbe59f640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb0054260 0x7fefb004a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7fefbe59f640 1 -- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefb000aa90 msgr2=0x7fefb004a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7fefbe59f640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefb000aa90 0x7fefb004a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7fefbe59f640 1 -- 172.21.15.116:0/3230948459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb804fe10 con 0x7fefb00df410 2024-02-18T11:48:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7fefbdd9e640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefb000aa90 0x7fefb004a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb00e3290 msgr2=0x7f9eb004f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb00e3290 0x7f9eb004f9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097a0 tx=0x7f9eb402b8b0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2269681371 shutdown_connections 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb0038090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2269681371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb00e3290 0x7f9eb004f9a0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9eb40097a0 tx=0x7f9eb402b8b0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.081+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2269681371 >> 172.21.15.116:0/2269681371 conn(0x7f9eb0009fe0 msgr2=0x7f9eb000a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.082+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2269681371 shutdown_connections 2024-02-18T11:48:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.082+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2269681371 wait complete. 2024-02-18T11:48:01.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.082+0000 7fefbe59f640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 0x7fefb00e17d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fefb8069730 tx=0x7fefb806aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.082+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c073260 con 0x7fc11c16a2a0 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.082+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc1140407f0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.083+0000 7fc118ff9640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc10806bc70 0x7fc10806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.083+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc1140ad020 con 0x7fc11c16a2a0 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.083+0000 7fc11bfff640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc10806bc70 0x7fc10806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.083+0000 7f9ebfe81640 1 Processor -- start 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.083+0000 7f9ebfe81640 1 -- start start 2024-02-18T11:48:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.083+0000 7fc11bfff640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc10806bc70 0x7fc10806e130 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0017a0 tx=0x7fc10c001970 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebfe81640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 0x7f9eb00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebfe81640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb00e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebfe81640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb017cdf0 0x7f9eb00e5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebfe81640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb017c100 con 0x7f9eb004fee0 2024-02-18T11:48:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebfe81640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb00d0440 con 0x7f9eb017cdf0 2024-02-18T11:48:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebfe81640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb017c310 con 0x7f9eb0054490 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 0x7f9eb00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 0x7f9eb00cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57540/0 (socket says 172.21.15.116:57540) 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2224890212 learned_addr learned my addr 172.21.15.116:0/2224890212 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebd41c640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb00e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebe41e640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb017cdf0 0x7f9eb00e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 msgr2=0x7f9eb00e4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb00e4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb017cdf0 msgr2=0x7f9eb00e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb017cdf0 0x7f9eb00e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebdc1d640 1 -- 172.21.15.116:0/2224890212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009400 con 0x7f9eb004fee0 2024-02-18T11:48:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.084+0000 7f9ebd41c640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb00e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:01.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.085+0000 7f9ebdc1d640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 0x7f9eb00cccc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9eb402be40 tx=0x7f9eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.087+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3790077690 con 0x7f379c100cf0 2024-02-18T11:48:01.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.087+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1140e08c0 con 0x7fc11c16a2a0 2024-02-18T11:48:01.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7f40099050 con 0x7f7f3804f280 2024-02-18T11:48:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33403c040 con 0x7fa33c04fee0 2024-02-18T11:48:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb80af890 con 0x7fefb00df410 2024-02-18T11:48:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb80af9f0 con 0x7fefb00df410 2024-02-18T11:48:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb80afd60 con 0x7fefb00df410 2024-02-18T11:48:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa334037c90 con 0x7fa33c04fee0 2024-02-18T11:48:01.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7f7f3d7fa640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f2406bcc0 0x7f7f2406e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.091+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7f40116320 con 0x7f7f3804f280 2024-02-18T11:48:01.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb004afe0 con 0x7fefb00df410 2024-02-18T11:48:01.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.090+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb00d6d40 con 0x7fefb00df410 2024-02-18T11:48:01.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.091+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33402f850 con 0x7fa33c04fee0 2024-02-18T11:48:01.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.091+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3340094d0 con 0x7fa33c04fee0 2024-02-18T11:48:01.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c049e40 con 0x7fa33c04fee0 2024-02-18T11:48:01.132 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.091+0000 7f7f3ffff640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f2406bcc0 0x7f7f2406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7f7f3ffff640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f2406bcc0 0x7f7f2406e180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7f38038f20 tx=0x7f7f2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb403ad60 con 0x7f9eb004fee0 2024-02-18T11:48:01.133 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb004f8f0 con 0x7fefb00df410 2024-02-18T11:48:01.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb4044400 con 0x7f9eb004fee0 2024-02-18T11:48:01.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb017c5f0 con 0x7f9eb004fee0 2024-02-18T11:48:01.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.092+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb00cd200 con 0x7f9eb004fee0 2024-02-18T11:48:01.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.093+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb40447b0 con 0x7f9eb004fee0 2024-02-18T11:48:01.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.094+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0005290 con 0x7f9eb004fee0 2024-02-18T11:48:01.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.095+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c04f8f0 con 0x7fa33c04fee0 2024-02-18T11:48:01.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.098+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f400e0930 con 0x7f7f3804f280 2024-02-18T11:48:01.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.100+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fefb80b6040 con 0x7fefb00df410 2024-02-18T11:48:01.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.101+0000 7fefae7fc640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef9c06bcc0 0x7fef9c06e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.101+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9eb404d440 con 0x7f9eb004fee0 2024-02-18T11:48:01.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.102+0000 7f9eae7fc640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9e9c06bd40 0x7f9e9c06e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.102+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9eb40ad200 con 0x7f9eb004fee0 2024-02-18T11:48:01.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.101+0000 7fefbdd9e640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef9c06bcc0 0x7fef9c06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.103+0000 7fefbdd9e640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef9c06bcc0 0x7fef9c06e180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007d90 tx=0x7fefb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.103+0000 7f9ebd41c640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9e9c06bd40 0x7f9e9c06e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.105+0000 7f9ebd41c640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9e9c06bd40 0x7f9e9c06e200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00e5cb0 tx=0x7f9ea8007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.104+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa3340423f0 con 0x7fa33c04fee0 2024-02-18T11:48:01.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.105+0000 7fa32b7fe640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa33006bca0 0x7fa33006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.137 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.105+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa3340ad260 con 0x7fa33c04fee0 2024-02-18T11:48:01.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.105+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fefb80f57d0 con 0x7fefb00df410 2024-02-18T11:48:01.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.105+0000 7fa342ffd640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa33006bca0 0x7fa33006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.106+0000 7fa342ffd640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa33006bca0 0x7fa33006e160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa344069730 tx=0x7fa344068930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.111+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefb80f1a70 con 0x7fefb00df410 2024-02-18T11:48:01.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.121+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9eb4077730 con 0x7f9eb004fee0 2024-02-18T11:48:01.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.130+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa334077820 con 0x7fa33c04fee0 2024-02-18T11:48:01.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.425+0000 7f3bf19cc640 1 -- 172.21.15.116:0/1026695715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3bcc005540 con 0x7f3bec165cf0 2024-02-18T11:48:01.422 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.426+0000 7f3be97fa640 1 -- 172.21.15.116:0/1026695715 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3be407b100 con 0x7f3bec165cf0 2024-02-18T11:48:01.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.429+0000 7f3bd2ffd640 1 -- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3bd806bda0 msgr2=0x7f3bd806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.429+0000 7f3bd2ffd640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3bd806bda0 0x7f3bd806e260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3be0006fd0 tx=0x7f3be0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.429+0000 7f3bd2ffd640 1 -- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 msgr2=0x7f3bec0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.429+0000 7f3bd2ffd640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0ce540 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3be4036040 tx=0x7f3be4004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 -- 172.21.15.116:0/1026695715 shutdown_connections 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3bd806bda0 0x7f3bd806e260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bec165cf0 0x7f3bec0ce540 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3bec1658b0 0x7f3bec1669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 --2- 172.21.15.116:0/1026695715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bec147ee0 0x7f3bec138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 -- 172.21.15.116:0/1026695715 >> 172.21.15.116:0/1026695715 conn(0x7f3bec0b12b0 msgr2=0x7f3bec0cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.430+0000 7f3bd2ffd640 1 -- 172.21.15.116:0/1026695715 shutdown_connections 2024-02-18T11:48:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.431+0000 7f3bd2ffd640 1 -- 172.21.15.116:0/1026695715 wait complete. 2024-02-18T11:48:01.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-02-18T11:48:01.440 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.459+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3784005540 con 0x7f379c100cf0 2024-02-18T11:48:01.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.460+0000 7f37a0ff9640 1 -- 172.21.15.116:0/3541985196 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f379007ad50 con 0x7f379c100cf0 2024-02-18T11:48:01.455 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.463+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f378806bca0 msgr2=0x7f378806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.463+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f378806bca0 0x7f378806e160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f379800bac0 tx=0x7f3798008040 comp rx=0 tx=0).stop 2024-02-18T11:48:01.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.463+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 msgr2=0x7f379c02b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.463+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 0x7f379c02b4a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f379003e040 tx=0x7f37900046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 shutdown_connections 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f378806bca0 0x7f378806e160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f379c104970 0x7f379c04bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f379c100cf0 0x7f379c02b4a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 --2- 172.21.15.116:0/3541985196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f379c005ec0 0x7f379c0fe0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 >> 172.21.15.116:0/3541985196 conn(0x7f379c009fe0 msgr2=0x7f379c0359b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 shutdown_connections 2024-02-18T11:48:01.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.464+0000 7f37aaa0e640 1 -- 172.21.15.116:0/3541985196 wait complete. 2024-02-18T11:48:01.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.477+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc11c16da00 con 0x7fc11c16a2a0 2024-02-18T11:48:01.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.478+0000 7fc118ff9640 1 -- 172.21.15.116:0/3500278780 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc114039090 con 0x7fc11c16a2a0 2024-02-18T11:48:01.471 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-02-18T11:48:01.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.480+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9a7c074520 con 0x7f9a7c06bc70 2024-02-18T11:48:01.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.480+0000 7f9a727fc640 1 -- 172.21.15.116:0/1265778840 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a78058240 con 0x7f9a7c06bc70 2024-02-18T11:48:01.474 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.474 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc10806bc70 msgr2=0x7fc10806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc10806bc70 0x7fc10806e130 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0017a0 tx=0x7fc10c001970 comp rx=0 tx=0).stop 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c16a2a0 msgr2=0x7fc11c162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c16a2a0 0x7fc11c162670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc114033000 tx=0x7fc114004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 shutdown_connections 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc10806bc70 0x7fc10806e130 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.481+0000 7fc12121f640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc11c16a2a0 0x7fc11c162670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.482+0000 7fc12121f640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc11c0cbe10 0x7fc11c162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.482+0000 7fc12121f640 1 --2- 172.21.15.116:0/3500278780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc11c0b25b0 0x7fc11c16a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.482+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 >> 172.21.15.116:0/3500278780 conn(0x7fc11c0b12b0 msgr2=0x7fc11c0cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.482+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 shutdown_connections 2024-02-18T11:48:01.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.482+0000 7fc12121f640 1 -- 172.21.15.116:0/3500278780 wait complete. 2024-02-18T11:48:01.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a6006bd40 msgr2=0x7f9a6006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a6006bd40 0x7f9a6006e200 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0d15b0 tx=0x7f9a6c009040 comp rx=0 tx=0).stop 2024-02-18T11:48:01.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c06bc70 msgr2=0x7f9a7c0c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c06bc70 0x7f9a7c0c87d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800b470 tx=0x7f9a7800b930 comp rx=0 tx=0).stop 2024-02-18T11:48:01.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 shutdown_connections 2024-02-18T11:48:01.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a6006bd40 0x7f9a6006e200 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a7c1621f0 0x7f9a7c08a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a7c0c4de0 0x7f9a7c0cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 --2- 172.21.15.116:0/1265778840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a7c06bc70 0x7f9a7c0c87d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 >> 172.21.15.116:0/1265778840 conn(0x7f9a7c0b31f0 msgr2=0x7f9a7c0ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 shutdown_connections 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.483+0000 7f9a840eb640 1 -- 172.21.15.116:0/1265778840 wait complete. 2024-02-18T11:48:01.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.485+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa33c0089c0 con 0x7fa33c04fee0 2024-02-18T11:48:01.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.485+0000 7fa32b7fe640 1 -- 172.21.15.116:0/2181425689 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa334047020 con 0x7fa33c04fee0 2024-02-18T11:48:01.479 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa33006bca0 msgr2=0x7fa33006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa33006bca0 0x7fa33006e160 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa344069730 tx=0x7fa344068930 comp rx=0 tx=0).stop 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 msgr2=0x7fa33c0da8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c0da8c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa334033000 tx=0x7fa334004730 comp rx=0 tx=0).stop 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 shutdown_connections 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa33006bca0 0x7fa33006e160 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa33c04fee0 0x7fa33c0da8c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa33c04fa80 0x7fa33c0e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 --2- 172.21.15.116:0/2181425689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa33c00aa90 0x7fa33c0d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 >> 172.21.15.116:0/2181425689 conn(0x7fa33c009fa0 msgr2=0x7fa33c053e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 shutdown_connections 2024-02-18T11:48:01.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.488+0000 7fa349725640 1 -- 172.21.15.116:0/2181425689 wait complete. 2024-02-18T11:48:01.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.495+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f20005810 con 0x7f7f3804f280 2024-02-18T11:48:01.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.495+0000 7f7f3d7fa640 1 -- 172.21.15.116:0/436104119 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f400e3ff0 con 0x7f7f3804f280 2024-02-18T11:48:01.488 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f2406bcc0 msgr2=0x7f7f2406e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f2406bcc0 0x7f7f2406e180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7f38038f20 tx=0x7f7f2c009040 comp rx=0 tx=0).stop 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 msgr2=0x7f7f38194650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 0x7f7f38194650 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7f4004f440 tx=0x7f7f400741f0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 shutdown_connections 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7f2406bcc0 0x7f7f2406e180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f38100d90 0x7f7f3804e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.497+0000 7f7f4719a640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7f38053d10 0x7f7f380fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.498+0000 7f7f4719a640 1 --2- 172.21.15.116:0/436104119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f3804f280 0x7f7f38194650 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.498+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 >> 172.21.15.116:0/436104119 conn(0x7f7f38009fa0 msgr2=0x7f7f38039aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.498+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 shutdown_connections 2024-02-18T11:48:01.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.498+0000 7f7f4719a640 1 -- 172.21.15.116:0/436104119 wait complete. 2024-02-18T11:48:01.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-02-18T11:48:01.495 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-02-18T11:48:01.496 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-02-18T11:48:01.497 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.506+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fefb004b500 con 0x7fefb00df410 2024-02-18T11:48:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.506+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ea0005810 con 0x7f9eb004fee0 2024-02-18T11:48:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.506+0000 7fefae7fc640 1 -- 172.21.15.116:0/3230948459 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fefb80b4020 con 0x7fefb00df410 2024-02-18T11:48:01.501 INFO:teuthology.orchestra.run.smithi116.stdout:55834574858 2024-02-18T11:48:01.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.506+0000 7f9eae7fc640 1 -- 172.21.15.116:0/2224890212 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9eb407adf0 con 0x7f9eb004fee0 2024-02-18T11:48:01.502 INFO:teuthology.orchestra.run.smithi116.stdout:55834574859 2024-02-18T11:48:01.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef9c06bcc0 msgr2=0x7fef9c06e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.503 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef9c06bcc0 0x7fef9c06e180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fefb4007d90 tx=0x7fefb4008040 comp rx=0 tx=0).stop 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 msgr2=0x7fefb00e17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 0x7fefb00e17d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fefb8069730 tx=0x7fefb806aa80 comp rx=0 tx=0).stop 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 shutdown_connections 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef9c06bcc0 0x7fef9c06e180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefb00df410 0x7fefb00e17d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fefb0054260 0x7fefb004a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 --2- 172.21.15.116:0/3230948459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefb000aa90 0x7fefb004a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 >> 172.21.15.116:0/3230948459 conn(0x7fefb0009fa0 msgr2=0x7fefb00de2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 shutdown_connections 2024-02-18T11:48:01.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.508+0000 7fefbeda0640 1 -- 172.21.15.116:0/3230948459 wait complete. 2024-02-18T11:48:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9e9c06bd40 msgr2=0x7f9e9c06e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9e9c06bd40 0x7f9e9c06e200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00e5cb0 tx=0x7f9ea8007e80 comp rx=0 tx=0).stop 2024-02-18T11:48:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 msgr2=0x7f9eb00cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 0x7f9eb00cccc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9eb402be40 tx=0x7f9eb40046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 shutdown_connections 2024-02-18T11:48:01.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9e9c06bd40 0x7f9e9c06e200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9eb017cdf0 0x7f9eb00e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9eb0054490 0x7f9eb00e4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 --2- 172.21.15.116:0/2224890212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb004fee0 0x7f9eb00cccc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 >> 172.21.15.116:0/2224890212 conn(0x7f9eb0009fe0 msgr2=0x7f9eb0035ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 shutdown_connections 2024-02-18T11:48:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.509+0000 7f9ebfe81640 1 -- 172.21.15.116:0/2224890212 wait complete. 2024-02-18T11:48:01.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-02-18T11:48:01.513 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-02-18T11:48:01.518 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-02-18T11:48:01.519 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:01.522 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-02-18T11:48:01.523 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:01.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.672+0000 7f42347e2640 1 Processor -- start 2024-02-18T11:48:01.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.672+0000 7f42347e2640 1 -- start start 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f42347e2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f422c1611e0 0x7f422c0b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f42347e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c0cea00 0x7f422c06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f42347e2640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c0d1fa0 con 0x7f422c0cee60 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f42347e2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c0b7030 con 0x7f422c1611e0 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f42347e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0d2560 con 0x7f422c0cea00 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f423257e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f422c1611e0 0x7f422c0b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f423257e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f422c1611e0 0x7f422c0b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57554/0 (socket says 172.21.15.116:57554) 2024-02-18T11:48:01.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.673+0000 7f423257e640 1 -- 172.21.15.116:0/137388401 learned_addr learned my addr 172.21.15.116:0/137388401 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f4231d7d640 1 --2- 172.21.15.116:0/137388401 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c0cea00 0x7f422c06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f423257e640 1 -- 172.21.15.116:0/137388401 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c0cea00 msgr2=0x7f422c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:01.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f423257e640 1 --2- 172.21.15.116:0/137388401 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c0cea00 0x7f422c06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f423257e640 1 -- 172.21.15.116:0/137388401 >> v1:172.21.15.143:6789/0 conn(0x7f422c0cee60 legacy=0x7f422c0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f423257e640 1 -- 172.21.15.116:0/137388401 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c0d4740 con 0x7f422c1611e0 2024-02-18T11:48:01.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f423257e640 1 --2- 172.21.15.116:0/137388401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c1611e0 0x7f422c0b6af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4228009e80 tx=0x7f422802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0aa488e545cf929 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.674+0000 7f423157c640 1 -- 172.21.15.116:0/137388401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422802caf0 con 0x7f422c1611e0 2024-02-18T11:48:01.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f423157c640 1 -- 172.21.15.116:0/137388401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422802cc50 con 0x7f422c1611e0 2024-02-18T11:48:01.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f423157c640 1 -- 172.21.15.116:0/137388401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42280343f0 con 0x7f422c1611e0 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 -- 172.21.15.116:0/137388401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c1611e0 msgr2=0x7f422c0b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 --2- 172.21.15.116:0/137388401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c1611e0 0x7f422c0b6af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4228009e80 tx=0x7f422802bf50 comp rx=0 tx=0).stop 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 -- 172.21.15.116:0/137388401 shutdown_connections 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 --2- 172.21.15.116:0/137388401 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c0cea00 0x7f422c06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 --2- 172.21.15.116:0/137388401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c1611e0 0x7f422c0b6af0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 -- 172.21.15.116:0/137388401 >> 172.21.15.116:0/137388401 conn(0x7f422c0b5b00 msgr2=0x7f422c0d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 -- 172.21.15.116:0/137388401 shutdown_connections 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.675+0000 7f42347e2640 1 -- 172.21.15.116:0/137388401 wait complete. 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.676+0000 7f42347e2640 1 Processor -- start 2024-02-18T11:48:01.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.676+0000 7f42347e2640 1 -- start start 2024-02-18T11:48:01.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f42347e2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f422c0cea00 0x7f422c0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f42347e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 0x7f422c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f42347e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c1611e0 0x7f422c0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f42347e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0c9980 con 0x7f422c0cee60 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f42347e2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0c8d70 con 0x7f422c0cea00 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f42347e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0c9050 con 0x7f422c1611e0 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 0x7f422c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 0x7f422c15cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57568/0 (socket says 172.21.15.116:57568) 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 -- 172.21.15.116:0/912661046 learned_addr learned my addr 172.21.15.116:0/912661046 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f423257e640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f422c0cea00 0x7f422c0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4232d7f640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c1611e0 0x7f422c0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 -- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c1611e0 msgr2=0x7f422c0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c1611e0 0x7f422c0c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 -- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f422c0cea00 msgr2=0x7f422c0c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f422c0cea00 0x7f422c0c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f4231d7d640 1 -- 172.21.15.116:0/912661046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228009b30 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.677+0000 7f423257e640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f422c0cea00 0x7f422c0c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.678+0000 7f4231d7d640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 0x7f422c15cf80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f421c00d5d0 tx=0x7f421c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.678+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0125f0 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.678+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c012770 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.678+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c0c9330 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.678+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c012b20 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.678+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c0c8a80 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.679+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c067840 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.679+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f421c012c80 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.679+0000 7f4222ffd640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f421006bc70 0x7f421006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.679+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f421c08afe0 con 0x7f422c0cee60 2024-02-18T11:48:01.674 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.679+0000 7f423257e640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f421006bc70 0x7f421006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:01.675 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.680+0000 7f423257e640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f421006bc70 0x7f421006e130 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4228035000 tx=0x7f4228030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:01.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.684+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f421c055480 con 0x7f422c0cee60 2024-02-18T11:48:01.843 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.849+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f422c0b2d30 con 0x7f421006bc70 2024-02-18T11:48:01.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.857+0000 7f4222ffd640 1 -- 172.21.15.116:0/912661046 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f4228044090 con 0x7f421006bc70 2024-02-18T11:48:01.854 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:01.856 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:01.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.860+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f421006bc70 msgr2=0x7f421006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.860+0000 7f42347e2640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f421006bc70 0x7f421006e130 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4228035000 tx=0x7f4228030000 comp rx=0 tx=0).stop 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.860+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 msgr2=0x7f422c15cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.860+0000 7f42347e2640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 0x7f422c15cf80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f421c00d5d0 tx=0x7f421c00da90 comp rx=0 tx=0).stop 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 shutdown_connections 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f421006bc70 0x7f421006e130 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f422c1611e0 0x7f422c0c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f422c0cee60 0x7f422c15cf80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 --2- 172.21.15.116:0/912661046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f422c0cea00 0x7f422c0c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 >> 172.21.15.116:0/912661046 conn(0x7f422c0b5b00 msgr2=0x7f422c160060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 shutdown_connections 2024-02-18T11:48:01.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:01.861+0000 7f42347e2640 1 -- 172.21.15.116:0/912661046 wait complete. 2024-02-18T11:48:01.865 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-02-18T11:48:00.295774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003251"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624689+0000","last_change":"2024-02-18T11:47:33.523482+0000","last_active":"2024-02-18T11:47:39.624689+0000","last_peered":"2024-02-18T11:47:39.624689+0000","last_clean":"2024-02-18T11:47:39.624689+0000","last_became_active":"2024-02-18T11:47:33.523129+0000","last_became_peered":"2024-02-18T11:47:33.523129+0000","last_unstale":"2024-02-18T11:47:39.624689+0000","last_undegraded":"2024-02-18T11:47:39.624689+0000","last_fullsized":"2024-02-18T11:47:39.624689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555924+0000","last_change":"2024-02-18T11:47:34.538612+0000","last_active":"2024-02-18T11:47:39.555924+0000","last_peered":"2024-02-18T11:47:39.555924+0000","last_clean":"2024-02-18T11:47:39.555924+0000","last_became_active":"2024-02-18T11:47:34.538315+0000","last_became_peered":"2024-02-18T11:47:34.538315+0000","last_unstale":"2024-02-18T11:47:39.555924+0000","last_undegraded":"2024-02-18T11:47:39.555924+0000","last_fullsized":"2024-02-18T11:47:39.555924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555044+0000","last_change":"2024-02-18T11:47:38.585025+0000","last_active":"2024-02-18T11:47:39.555044+0000","last_peered":"2024-02-18T11:47:39.555044+0000","last_clean":"2024-02-18T11:47:39.555044+0000","last_became_active":"2024-02-18T11:47:38.584669+0000","last_became_peered":"2024-02-18T11:47:38.584669+0000","last_unstale":"2024-02-18T11:47:39.555044+0000","last_undegraded":"2024-02-18T11:47:39.555044+0000","last_fullsized":"2024-02-18T11:47:39.555044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:07:55.638374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556025+0000","last_change":"2024-02-18T11:47:33.516291+0000","last_active":"2024-02-18T11:47:39.556025+0000","last_peered":"2024-02-18T11:47:39.556025+0000","last_clean":"2024-02-18T11:47:39.556025+0000","last_became_active":"2024-02-18T11:47:33.516043+0000","last_became_peered":"2024-02-18T11:47:33.516043+0000","last_unstale":"2024-02-18T11:47:39.556025+0000","last_undegraded":"2024-02-18T11:47:39.556025+0000","last_fullsized":"2024-02-18T11:47:39.556025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624545+0000","last_change":"2024-02-18T11:47:34.546759+0000","last_active":"2024-02-18T11:47:39.624545+0000","last_peered":"2024-02-18T11:47:39.624545+0000","last_clean":"2024-02-18T11:47:39.624545+0000","last_became_active":"2024-02-18T11:47:34.546282+0000","last_became_peered":"2024-02-18T11:47:34.546282+0000","last_unstale":"2024-02-18T11:47:39.624545+0000","last_undegraded":"2024-02-18T11:47:39.624545+0000","last_fullsized":"2024-02-18T11:47:39.624545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808452+0000","last_change":"2024-02-18T11:47:38.571574+0000","last_active":"2024-02-18T11:47:39.808452+0000","last_peered":"2024-02-18T11:47:39.808452+0000","last_clean":"2024-02-18T11:47:39.808452+0000","last_became_active":"2024-02-18T11:47:38.571177+0000","last_became_peered":"2024-02-18T11:47:38.571177+0000","last_unstale":"2024-02-18T11:47:39.808452+0000","last_undegraded":"2024-02-18T11:47:39.808452+0000","last_fullsized":"2024-02-18T11:47:39.808452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:29.976010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309592+0000","last_change":"2024-02-18T11:47:33.522407+0000","last_active":"2024-02-18T11:47:40.309592+0000","last_peered":"2024-02-18T11:47:40.309592+0000","last_clean":"2024-02-18T11:47:40.309592+0000","last_became_active":"2024-02-18T11:47:33.521982+0000","last_became_peered":"2024-02-18T11:47:33.521982+0000","last_unstale":"2024-02-18T11:47:40.309592+0000","last_undegraded":"2024-02-18T11:47:40.309592+0000","last_fullsized":"2024-02-18T11:47:40.309592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:54:58.132971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172176+0000","last_change":"2024-02-18T11:47:34.546074+0000","last_active":"2024-02-18T11:47:40.172176+0000","last_peered":"2024-02-18T11:47:40.172176+0000","last_clean":"2024-02-18T11:47:40.172176+0000","last_became_active":"2024-02-18T11:47:34.545447+0000","last_became_peered":"2024-02-18T11:47:34.545447+0000","last_unstale":"2024-02-18T11:47:40.172176+0000","last_undegraded":"2024-02-18T11:47:40.172176+0000","last_fullsized":"2024-02-18T11:47:40.172176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809128+0000","last_change":"2024-02-18T11:47:38.568989+0000","last_active":"2024-02-18T11:47:39.809128+0000","last_peered":"2024-02-18T11:47:39.809128+0000","last_clean":"2024-02-18T11:47:39.809128+0000","last_became_active":"2024-02-18T11:47:38.568524+0000","last_became_peered":"2024-02-18T11:47:38.568524+0000","last_unstale":"2024-02-18T11:47:39.809128+0000","last_undegraded":"2024-02-18T11:47:39.809128+0000","last_fullsized":"2024-02-18T11:47:39.809128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309327+0000","last_change":"2024-02-18T11:47:33.527284+0000","last_active":"2024-02-18T11:47:40.309327+0000","last_peered":"2024-02-18T11:47:40.309327+0000","last_clean":"2024-02-18T11:47:40.309327+0000","last_became_active":"2024-02-18T11:47:33.526669+0000","last_became_peered":"2024-02-18T11:47:33.526669+0000","last_unstale":"2024-02-18T11:47:40.309327+0000","last_undegraded":"2024-02-18T11:47:40.309327+0000","last_fullsized":"2024-02-18T11:47:40.309327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172388+0000","last_change":"2024-02-18T11:47:34.542045+0000","last_active":"2024-02-18T11:47:40.172388+0000","last_peered":"2024-02-18T11:47:40.172388+0000","last_clean":"2024-02-18T11:47:40.172388+0000","last_became_active":"2024-02-18T11:47:34.541776+0000","last_became_peered":"2024-02-18T11:47:34.541776+0000","last_unstale":"2024-02-18T11:47:40.172388+0000","last_undegraded":"2024-02-18T11:47:40.172388+0000","last_fullsized":"2024-02-18T11:47:40.172388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808568+0000","last_change":"2024-02-18T11:47:38.570454+0000","last_active":"2024-02-18T11:47:39.808568+0000","last_peered":"2024-02-18T11:47:39.808568+0000","last_clean":"2024-02-18T11:47:39.808568+0000","last_became_active":"2024-02-18T11:47:38.569640+0000","last_became_peered":"2024-02-18T11:47:38.569640+0000","last_unstale":"2024-02-18T11:47:39.808568+0000","last_undegraded":"2024-02-18T11:47:39.808568+0000","last_fullsized":"2024-02-18T11:47:39.808568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:36.793248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556080+0000","last_change":"2024-02-18T11:47:34.534695+0000","last_active":"2024-02-18T11:47:39.556080+0000","last_peered":"2024-02-18T11:47:39.556080+0000","last_clean":"2024-02-18T11:47:39.556080+0000","last_became_active":"2024-02-18T11:47:34.534368+0000","last_became_peered":"2024-02-18T11:47:34.534368+0000","last_unstale":"2024-02-18T11:47:39.556080+0000","last_undegraded":"2024-02-18T11:47:39.556080+0000","last_fullsized":"2024-02-18T11:47:39.556080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309372+0000","last_change":"2024-02-18T11:47:33.521962+0000","last_active":"2024-02-18T11:47:40.309372+0000","last_peered":"2024-02-18T11:47:40.309372+0000","last_clean":"2024-02-18T11:47:40.309372+0000","last_became_active":"2024-02-18T11:47:33.521649+0000","last_became_peered":"2024-02-18T11:47:33.521649+0000","last_unstale":"2024-02-18T11:47:40.309372+0000","last_undegraded":"2024-02-18T11:47:40.309372+0000","last_fullsized":"2024-02-18T11:47:40.309372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:37.645450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808951+0000","last_change":"2024-02-18T11:47:38.571998+0000","last_active":"2024-02-18T11:47:39.808951+0000","last_peered":"2024-02-18T11:47:39.808951+0000","last_clean":"2024-02-18T11:47:39.808951+0000","last_became_active":"2024-02-18T11:47:38.571568+0000","last_became_peered":"2024-02-18T11:47:38.571568+0000","last_unstale":"2024-02-18T11:47:39.808951+0000","last_undegraded":"2024-02-18T11:47:39.808951+0000","last_fullsized":"2024-02-18T11:47:39.808951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:47.822731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555491+0000","last_change":"2024-02-18T11:47:34.543371+0000","last_active":"2024-02-18T11:47:39.555491+0000","last_peered":"2024-02-18T11:47:39.555491+0000","last_clean":"2024-02-18T11:47:39.555491+0000","last_became_active":"2024-02-18T11:47:34.542194+0000","last_became_peered":"2024-02-18T11:47:34.542194+0000","last_unstale":"2024-02-18T11:47:39.555491+0000","last_undegraded":"2024-02-18T11:47:39.555491+0000","last_fullsized":"2024-02-18T11:47:39.555491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845296+0000","last_change":"2024-02-18T11:47:33.517942+0000","last_active":"2024-02-18T11:47:39.845296+0000","last_peered":"2024-02-18T11:47:39.845296+0000","last_clean":"2024-02-18T11:47:39.845296+0000","last_became_active":"2024-02-18T11:47:33.517664+0000","last_became_peered":"2024-02-18T11:47:33.517664+0000","last_unstale":"2024-02-18T11:47:39.845296+0000","last_undegraded":"2024-02-18T11:47:39.845296+0000","last_fullsized":"2024-02-18T11:47:39.845296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808749+0000","last_change":"2024-02-18T11:47:38.573311+0000","last_active":"2024-02-18T11:47:39.808749+0000","last_peered":"2024-02-18T11:47:39.808749+0000","last_clean":"2024-02-18T11:47:39.808749+0000","last_became_active":"2024-02-18T11:47:38.572888+0000","last_became_peered":"2024-02-18T11:47:38.572888+0000","last_unstale":"2024-02-18T11:47:39.808749+0000","last_undegraded":"2024-02-18T11:47:39.808749+0000","last_fullsized":"2024-02-18T11:47:39.808749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:51.214710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555516+0000","last_change":"2024-02-18T11:47:34.537515+0000","last_active":"2024-02-18T11:47:39.555516+0000","last_peered":"2024-02-18T11:47:39.555516+0000","last_clean":"2024-02-18T11:47:39.555516+0000","last_became_active":"2024-02-18T11:47:34.537235+0000","last_became_peered":"2024-02-18T11:47:34.537235+0000","last_unstale":"2024-02-18T11:47:39.555516+0000","last_undegraded":"2024-02-18T11:47:39.555516+0000","last_fullsized":"2024-02-18T11:47:39.555516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844871+0000","last_change":"2024-02-18T11:47:33.522206+0000","last_active":"2024-02-18T11:47:39.844871+0000","last_peered":"2024-02-18T11:47:39.844871+0000","last_clean":"2024-02-18T11:47:39.844871+0000","last_became_active":"2024-02-18T11:47:33.521878+0000","last_became_peered":"2024-02-18T11:47:33.521878+0000","last_unstale":"2024-02-18T11:47:39.844871+0000","last_undegraded":"2024-02-18T11:47:39.844871+0000","last_fullsized":"2024-02-18T11:47:39.844871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845354+0000","last_change":"2024-02-18T11:47:38.584826+0000","last_active":"2024-02-18T11:47:39.845354+0000","last_peered":"2024-02-18T11:47:39.845354+0000","last_clean":"2024-02-18T11:47:39.845354+0000","last_became_active":"2024-02-18T11:47:38.584161+0000","last_became_peered":"2024-02-18T11:47:38.584161+0000","last_unstale":"2024-02-18T11:47:39.845354+0000","last_undegraded":"2024-02-18T11:47:39.845354+0000","last_fullsized":"2024-02-18T11:47:39.845354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808871+0000","last_change":"2024-02-18T11:47:34.533256+0000","last_active":"2024-02-18T11:47:39.808871+0000","last_peered":"2024-02-18T11:47:39.808871+0000","last_clean":"2024-02-18T11:47:39.808871+0000","last_became_active":"2024-02-18T11:47:34.531682+0000","last_became_peered":"2024-02-18T11:47:34.531682+0000","last_unstale":"2024-02-18T11:47:39.808871+0000","last_undegraded":"2024-02-18T11:47:39.808871+0000","last_fullsized":"2024-02-18T11:47:39.808871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:09.297073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308926+0000","last_change":"2024-02-18T11:47:33.523406+0000","last_active":"2024-02-18T11:47:40.308926+0000","last_peered":"2024-02-18T11:47:40.308926+0000","last_clean":"2024-02-18T11:47:40.308926+0000","last_became_active":"2024-02-18T11:47:33.522929+0000","last_became_peered":"2024-02-18T11:47:33.522929+0000","last_unstale":"2024-02-18T11:47:40.308926+0000","last_undegraded":"2024-02-18T11:47:40.308926+0000","last_fullsized":"2024-02-18T11:47:40.308926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809106+0000","last_change":"2024-02-18T11:47:38.572268+0000","last_active":"2024-02-18T11:47:39.809106+0000","last_peered":"2024-02-18T11:47:39.809106+0000","last_clean":"2024-02-18T11:47:39.809106+0000","last_became_active":"2024-02-18T11:47:38.571698+0000","last_became_peered":"2024-02-18T11:47:38.571698+0000","last_unstale":"2024-02-18T11:47:39.809106+0000","last_undegraded":"2024-02-18T11:47:39.809106+0000","last_fullsized":"2024-02-18T11:47:39.809106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:59.447791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624439+0000","last_change":"2024-02-18T11:47:34.537052+0000","last_active":"2024-02-18T11:47:39.624439+0000","last_peered":"2024-02-18T11:47:39.624439+0000","last_clean":"2024-02-18T11:47:39.624439+0000","last_became_active":"2024-02-18T11:47:34.536782+0000","last_became_peered":"2024-02-18T11:47:34.536782+0000","last_unstale":"2024-02-18T11:47:39.624439+0000","last_undegraded":"2024-02-18T11:47:39.624439+0000","last_fullsized":"2024-02-18T11:47:39.624439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556919+0000","last_change":"2024-02-18T11:47:33.519918+0000","last_active":"2024-02-18T11:47:39.556919+0000","last_peered":"2024-02-18T11:47:39.556919+0000","last_clean":"2024-02-18T11:47:39.556919+0000","last_became_active":"2024-02-18T11:47:33.519651+0000","last_became_peered":"2024-02-18T11:47:33.519651+0000","last_unstale":"2024-02-18T11:47:39.556919+0000","last_undegraded":"2024-02-18T11:47:39.556919+0000","last_fullsized":"2024-02-18T11:47:39.556919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808561+0000","last_change":"2024-02-18T11:47:38.584734+0000","last_active":"2024-02-18T11:47:39.808561+0000","last_peered":"2024-02-18T11:47:39.808561+0000","last_clean":"2024-02-18T11:47:39.808561+0000","last_became_active":"2024-02-18T11:47:38.584299+0000","last_became_peered":"2024-02-18T11:47:38.584299+0000","last_unstale":"2024-02-18T11:47:39.808561+0000","last_undegraded":"2024-02-18T11:47:39.808561+0000","last_fullsized":"2024-02-18T11:47:39.808561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:59.168505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555556+0000","last_change":"2024-02-18T11:47:34.550205+0000","last_active":"2024-02-18T11:47:39.555556+0000","last_peered":"2024-02-18T11:47:39.555556+0000","last_clean":"2024-02-18T11:47:39.555556+0000","last_became_active":"2024-02-18T11:47:34.549966+0000","last_became_peered":"2024-02-18T11:47:34.549966+0000","last_unstale":"2024-02-18T11:47:39.555556+0000","last_undegraded":"2024-02-18T11:47:39.555556+0000","last_fullsized":"2024-02-18T11:47:39.555556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845365+0000","last_change":"2024-02-18T11:47:33.513489+0000","last_active":"2024-02-18T11:47:39.845365+0000","last_peered":"2024-02-18T11:47:39.845365+0000","last_clean":"2024-02-18T11:47:39.845365+0000","last_became_active":"2024-02-18T11:47:33.513181+0000","last_became_peered":"2024-02-18T11:47:33.513181+0000","last_unstale":"2024-02-18T11:47:39.845365+0000","last_undegraded":"2024-02-18T11:47:39.845365+0000","last_fullsized":"2024-02-18T11:47:39.845365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554976+0000","last_change":"2024-02-18T11:47:38.582725+0000","last_active":"2024-02-18T11:47:39.554976+0000","last_peered":"2024-02-18T11:47:39.554976+0000","last_clean":"2024-02-18T11:47:39.554976+0000","last_became_active":"2024-02-18T11:47:38.582354+0000","last_became_peered":"2024-02-18T11:47:38.582354+0000","last_unstale":"2024-02-18T11:47:39.554976+0000","last_undegraded":"2024-02-18T11:47:39.554976+0000","last_fullsized":"2024-02-18T11:47:39.554976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845190+0000","last_change":"2024-02-18T11:47:34.543899+0000","last_active":"2024-02-18T11:47:39.845190+0000","last_peered":"2024-02-18T11:47:39.845190+0000","last_clean":"2024-02-18T11:47:39.845190+0000","last_became_active":"2024-02-18T11:47:34.543627+0000","last_became_peered":"2024-02-18T11:47:34.543627+0000","last_unstale":"2024-02-18T11:47:39.845190+0000","last_undegraded":"2024-02-18T11:47:39.845190+0000","last_fullsized":"2024-02-18T11:47:39.845190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309676+0000","last_change":"2024-02-18T11:47:33.523004+0000","last_active":"2024-02-18T11:47:40.309676+0000","last_peered":"2024-02-18T11:47:40.309676+0000","last_clean":"2024-02-18T11:47:40.309676+0000","last_became_active":"2024-02-18T11:47:33.522170+0000","last_became_peered":"2024-02-18T11:47:33.522170+0000","last_unstale":"2024-02-18T11:47:40.309676+0000","last_undegraded":"2024-02-18T11:47:40.309676+0000","last_fullsized":"2024-02-18T11:47:40.309676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:12.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624145+0000","last_change":"2024-02-18T11:47:38.582313+0000","last_active":"2024-02-18T11:47:39.624145+0000","last_peered":"2024-02-18T11:47:39.624145+0000","last_clean":"2024-02-18T11:47:39.624145+0000","last_became_active":"2024-02-18T11:47:38.581439+0000","last_became_peered":"2024-02-18T11:47:38.581439+0000","last_unstale":"2024-02-18T11:47:39.624145+0000","last_undegraded":"2024-02-18T11:47:39.624145+0000","last_fullsized":"2024-02-18T11:47:39.624145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172507+0000","last_change":"2024-02-18T11:47:34.546405+0000","last_active":"2024-02-18T11:47:40.172507+0000","last_peered":"2024-02-18T11:47:40.172507+0000","last_clean":"2024-02-18T11:47:40.172507+0000","last_became_active":"2024-02-18T11:47:34.545110+0000","last_became_peered":"2024-02-18T11:47:34.545110+0000","last_unstale":"2024-02-18T11:47:40.172507+0000","last_undegraded":"2024-02-18T11:47:40.172507+0000","last_fullsized":"2024-02-18T11:47:40.172507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845298+0000","last_change":"2024-02-18T11:47:33.515238+0000","last_active":"2024-02-18T11:47:39.845298+0000","last_peered":"2024-02-18T11:47:39.845298+0000","last_clean":"2024-02-18T11:47:39.845298+0000","last_became_active":"2024-02-18T11:47:33.514792+0000","last_became_peered":"2024-02-18T11:47:33.514792+0000","last_unstale":"2024-02-18T11:47:39.845298+0000","last_undegraded":"2024-02-18T11:47:39.845298+0000","last_fullsized":"2024-02-18T11:47:39.845298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845086+0000","last_change":"2024-02-18T11:47:38.569586+0000","last_active":"2024-02-18T11:47:39.845086+0000","last_peered":"2024-02-18T11:47:39.845086+0000","last_clean":"2024-02-18T11:47:39.845086+0000","last_became_active":"2024-02-18T11:47:38.568487+0000","last_became_peered":"2024-02-18T11:47:38.568487+0000","last_unstale":"2024-02-18T11:47:39.845086+0000","last_undegraded":"2024-02-18T11:47:39.845086+0000","last_fullsized":"2024-02-18T11:47:39.845086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809233+0000","last_change":"2024-02-18T11:47:34.533351+0000","last_active":"2024-02-18T11:47:39.809233+0000","last_peered":"2024-02-18T11:47:39.809233+0000","last_clean":"2024-02-18T11:47:39.809233+0000","last_became_active":"2024-02-18T11:47:34.532667+0000","last_became_peered":"2024-02-18T11:47:34.532667+0000","last_unstale":"2024-02-18T11:47:39.809233+0000","last_undegraded":"2024-02-18T11:47:39.809233+0000","last_fullsized":"2024-02-18T11:47:39.809233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:26.230805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172223+0000","last_change":"2024-02-18T11:47:33.526859+0000","last_active":"2024-02-18T11:47:40.172223+0000","last_peered":"2024-02-18T11:47:40.172223+0000","last_clean":"2024-02-18T11:47:40.172223+0000","last_became_active":"2024-02-18T11:47:33.525839+0000","last_became_peered":"2024-02-18T11:47:33.525839+0000","last_unstale":"2024-02-18T11:47:40.172223+0000","last_undegraded":"2024-02-18T11:47:40.172223+0000","last_fullsized":"2024-02-18T11:47:40.172223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845416+0000","last_change":"2024-02-18T11:47:38.562544+0000","last_active":"2024-02-18T11:47:39.845416+0000","last_peered":"2024-02-18T11:47:39.845416+0000","last_clean":"2024-02-18T11:47:39.845416+0000","last_became_active":"2024-02-18T11:47:38.562171+0000","last_became_peered":"2024-02-18T11:47:38.562171+0000","last_unstale":"2024-02-18T11:47:39.845416+0000","last_undegraded":"2024-02-18T11:47:39.845416+0000","last_fullsized":"2024-02-18T11:47:39.845416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844931+0000","last_change":"2024-02-18T11:47:34.544098+0000","last_active":"2024-02-18T11:47:39.844931+0000","last_peered":"2024-02-18T11:47:39.844931+0000","last_clean":"2024-02-18T11:47:39.844931+0000","last_became_active":"2024-02-18T11:47:34.543279+0000","last_became_peered":"2024-02-18T11:47:34.543279+0000","last_unstale":"2024-02-18T11:47:39.844931+0000","last_undegraded":"2024-02-18T11:47:39.844931+0000","last_fullsized":"2024-02-18T11:47:39.844931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554630+0000","last_change":"2024-02-18T11:47:33.518923+0000","last_active":"2024-02-18T11:47:39.554630+0000","last_peered":"2024-02-18T11:47:39.554630+0000","last_clean":"2024-02-18T11:47:39.554630+0000","last_became_active":"2024-02-18T11:47:33.518591+0000","last_became_peered":"2024-02-18T11:47:33.518591+0000","last_unstale":"2024-02-18T11:47:39.554630+0000","last_undegraded":"2024-02-18T11:47:39.554630+0000","last_fullsized":"2024-02-18T11:47:39.554630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845451+0000","last_change":"2024-02-18T11:47:38.571577+0000","last_active":"2024-02-18T11:47:39.845451+0000","last_peered":"2024-02-18T11:47:39.845451+0000","last_clean":"2024-02-18T11:47:39.845451+0000","last_became_active":"2024-02-18T11:47:38.571165+0000","last_became_peered":"2024-02-18T11:47:38.571165+0000","last_unstale":"2024-02-18T11:47:39.845451+0000","last_undegraded":"2024-02-18T11:47:39.845451+0000","last_fullsized":"2024-02-18T11:47:39.845451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.623861+0000","last_change":"2024-02-18T11:47:34.533365+0000","last_active":"2024-02-18T11:47:39.623861+0000","last_peered":"2024-02-18T11:47:39.623861+0000","last_clean":"2024-02-18T11:47:39.623861+0000","last_became_active":"2024-02-18T11:47:34.533103+0000","last_became_peered":"2024-02-18T11:47:34.533103+0000","last_unstale":"2024-02-18T11:47:39.623861+0000","last_undegraded":"2024-02-18T11:47:39.623861+0000","last_fullsized":"2024-02-18T11:47:39.623861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:13.022192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308945+0000","last_change":"2024-02-18T11:47:33.523599+0000","last_active":"2024-02-18T11:47:40.308945+0000","last_peered":"2024-02-18T11:47:40.308945+0000","last_clean":"2024-02-18T11:47:40.308945+0000","last_became_active":"2024-02-18T11:47:33.523327+0000","last_became_peered":"2024-02-18T11:47:33.523327+0000","last_unstale":"2024-02-18T11:47:40.308945+0000","last_undegraded":"2024-02-18T11:47:40.308945+0000","last_fullsized":"2024-02-18T11:47:40.308945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624414+0000","last_change":"2024-02-18T11:47:38.581803+0000","last_active":"2024-02-18T11:47:39.624414+0000","last_peered":"2024-02-18T11:47:39.624414+0000","last_clean":"2024-02-18T11:47:39.624414+0000","last_became_active":"2024-02-18T11:47:38.581401+0000","last_became_peered":"2024-02-18T11:47:38.581401+0000","last_unstale":"2024-02-18T11:47:39.624414+0000","last_undegraded":"2024-02-18T11:47:39.624414+0000","last_fullsized":"2024-02-18T11:47:39.624414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556062+0000","last_change":"2024-02-18T11:47:34.544673+0000","last_active":"2024-02-18T11:47:39.556062+0000","last_peered":"2024-02-18T11:47:39.556062+0000","last_clean":"2024-02-18T11:47:39.556062+0000","last_became_active":"2024-02-18T11:47:34.544427+0000","last_became_peered":"2024-02-18T11:47:34.544427+0000","last_unstale":"2024-02-18T11:47:39.556062+0000","last_undegraded":"2024-02-18T11:47:39.556062+0000","last_fullsized":"2024-02-18T11:47:39.556062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172045+0000","last_change":"2024-02-18T11:47:33.522514+0000","last_active":"2024-02-18T11:47:40.172045+0000","last_peered":"2024-02-18T11:47:40.172045+0000","last_clean":"2024-02-18T11:47:40.172045+0000","last_became_active":"2024-02-18T11:47:33.522205+0000","last_became_peered":"2024-02-18T11:47:33.522205+0000","last_unstale":"2024-02-18T11:47:40.172045+0000","last_undegraded":"2024-02-18T11:47:40.172045+0000","last_fullsized":"2024-02-18T11:47:40.172045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309116+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:47:40.309116+0000","last_peered":"2024-02-18T11:47:40.309116+0000","last_clean":"2024-02-18T11:47:40.309116+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:47:40.309116+0000","last_undegraded":"2024-02-18T11:47:40.309116+0000","last_fullsized":"2024-02-18T11:47:40.309116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556628+0000","last_change":"2024-02-18T11:47:38.584484+0000","last_active":"2024-02-18T11:47:39.556628+0000","last_peered":"2024-02-18T11:47:39.556628+0000","last_clean":"2024-02-18T11:47:39.556628+0000","last_became_active":"2024-02-18T11:47:38.584089+0000","last_became_peered":"2024-02-18T11:47:38.584089+0000","last_unstale":"2024-02-18T11:47:39.556628+0000","last_undegraded":"2024-02-18T11:47:39.556628+0000","last_fullsized":"2024-02-18T11:47:39.556628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309440+0000","last_change":"2024-02-18T11:47:34.548586+0000","last_active":"2024-02-18T11:47:40.309440+0000","last_peered":"2024-02-18T11:47:40.309440+0000","last_clean":"2024-02-18T11:47:40.309440+0000","last_became_active":"2024-02-18T11:47:34.548315+0000","last_became_peered":"2024-02-18T11:47:34.548315+0000","last_unstale":"2024-02-18T11:47:40.309440+0000","last_undegraded":"2024-02-18T11:47:40.309440+0000","last_fullsized":"2024-02-18T11:47:40.309440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554890+0000","last_change":"2024-02-18T11:47:33.526959+0000","last_active":"2024-02-18T11:47:39.554890+0000","last_peered":"2024-02-18T11:47:39.554890+0000","last_clean":"2024-02-18T11:47:39.554890+0000","last_became_active":"2024-02-18T11:47:33.526632+0000","last_became_peered":"2024-02-18T11:47:33.526632+0000","last_unstale":"2024-02-18T11:47:39.554890+0000","last_undegraded":"2024-02-18T11:47:39.554890+0000","last_fullsized":"2024-02-18T11:47:39.554890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556483+0000","last_change":"2024-02-18T11:47:38.581910+0000","last_active":"2024-02-18T11:47:39.556483+0000","last_peered":"2024-02-18T11:47:39.556483+0000","last_clean":"2024-02-18T11:47:39.556483+0000","last_became_active":"2024-02-18T11:47:38.581463+0000","last_became_peered":"2024-02-18T11:47:38.581463+0000","last_unstale":"2024-02-18T11:47:39.556483+0000","last_undegraded":"2024-02-18T11:47:39.556483+0000","last_fullsized":"2024-02-18T11:47:39.556483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172330+0000","last_change":"2024-02-18T11:47:34.545863+0000","last_active":"2024-02-18T11:47:40.172330+0000","last_peered":"2024-02-18T11:47:40.172330+0000","last_clean":"2024-02-18T11:47:40.172330+0000","last_became_active":"2024-02-18T11:47:34.545142+0000","last_became_peered":"2024-02-18T11:47:34.545142+0000","last_unstale":"2024-02-18T11:47:40.172330+0000","last_undegraded":"2024-02-18T11:47:40.172330+0000","last_fullsized":"2024-02-18T11:47:40.172330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845452+0000","last_change":"2024-02-18T11:47:33.519784+0000","last_active":"2024-02-18T11:47:39.845452+0000","last_peered":"2024-02-18T11:47:39.845452+0000","last_clean":"2024-02-18T11:47:39.845452+0000","last_became_active":"2024-02-18T11:47:33.519495+0000","last_became_peered":"2024-02-18T11:47:33.519495+0000","last_unstale":"2024-02-18T11:47:39.845452+0000","last_undegraded":"2024-02-18T11:47:39.845452+0000","last_fullsized":"2024-02-18T11:47:39.845452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624233+0000","last_change":"2024-02-18T11:47:38.575385+0000","last_active":"2024-02-18T11:47:39.624233+0000","last_peered":"2024-02-18T11:47:39.624233+0000","last_clean":"2024-02-18T11:47:39.624233+0000","last_became_active":"2024-02-18T11:47:38.574974+0000","last_became_peered":"2024-02-18T11:47:38.574974+0000","last_unstale":"2024-02-18T11:47:39.624233+0000","last_undegraded":"2024-02-18T11:47:39.624233+0000","last_fullsized":"2024-02-18T11:47:39.624233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309750+0000","last_change":"2024-02-18T11:47:34.550657+0000","last_active":"2024-02-18T11:47:40.309750+0000","last_peered":"2024-02-18T11:47:40.309750+0000","last_clean":"2024-02-18T11:47:40.309750+0000","last_became_active":"2024-02-18T11:47:34.550286+0000","last_became_peered":"2024-02-18T11:47:34.550286+0000","last_unstale":"2024-02-18T11:47:40.309750+0000","last_undegraded":"2024-02-18T11:47:40.309750+0000","last_fullsized":"2024-02-18T11:47:40.309750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555138+0000","last_change":"2024-02-18T11:47:33.518941+0000","last_active":"2024-02-18T11:47:39.555138+0000","last_peered":"2024-02-18T11:47:39.555138+0000","last_clean":"2024-02-18T11:47:39.555138+0000","last_became_active":"2024-02-18T11:47:33.518644+0000","last_became_peered":"2024-02-18T11:47:33.518644+0000","last_unstale":"2024-02-18T11:47:39.555138+0000","last_undegraded":"2024-02-18T11:47:39.555138+0000","last_fullsized":"2024-02-18T11:47:39.555138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171756+0000","last_change":"2024-02-18T11:47:38.582832+0000","last_active":"2024-02-18T11:47:40.171756+0000","last_peered":"2024-02-18T11:47:40.171756+0000","last_clean":"2024-02-18T11:47:40.171756+0000","last_became_active":"2024-02-18T11:47:38.582436+0000","last_became_peered":"2024-02-18T11:47:38.582436+0000","last_unstale":"2024-02-18T11:47:40.171756+0000","last_undegraded":"2024-02-18T11:47:40.171756+0000","last_fullsized":"2024-02-18T11:47:40.171756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309643+0000","last_change":"2024-02-18T11:47:34.544607+0000","last_active":"2024-02-18T11:47:40.309643+0000","last_peered":"2024-02-18T11:47:40.309643+0000","last_clean":"2024-02-18T11:47:40.309643+0000","last_became_active":"2024-02-18T11:47:34.544257+0000","last_became_peered":"2024-02-18T11:47:34.544257+0000","last_unstale":"2024-02-18T11:47:40.309643+0000","last_undegraded":"2024-02-18T11:47:40.309643+0000","last_fullsized":"2024-02-18T11:47:40.309643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809144+0000","last_change":"2024-02-18T11:47:33.516233+0000","last_active":"2024-02-18T11:47:39.809144+0000","last_peered":"2024-02-18T11:47:39.809144+0000","last_clean":"2024-02-18T11:47:39.809144+0000","last_became_active":"2024-02-18T11:47:33.515830+0000","last_became_peered":"2024-02-18T11:47:33.515830+0000","last_unstale":"2024-02-18T11:47:39.809144+0000","last_undegraded":"2024-02-18T11:47:39.809144+0000","last_fullsized":"2024-02-18T11:47:39.809144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:43:01.501097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808439+0000","last_change":"2024-02-18T11:47:38.573381+0000","last_active":"2024-02-18T11:47:39.808439+0000","last_peered":"2024-02-18T11:47:39.808439+0000","last_clean":"2024-02-18T11:47:39.808439+0000","last_became_active":"2024-02-18T11:47:38.572824+0000","last_became_peered":"2024-02-18T11:47:38.572824+0000","last_unstale":"2024-02-18T11:47:39.808439+0000","last_undegraded":"2024-02-18T11:47:39.808439+0000","last_fullsized":"2024-02-18T11:47:39.808439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:11.073343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309074+0000","last_change":"2024-02-18T11:47:34.543813+0000","last_active":"2024-02-18T11:47:40.309074+0000","last_peered":"2024-02-18T11:47:40.309074+0000","last_clean":"2024-02-18T11:47:40.309074+0000","last_became_active":"2024-02-18T11:47:34.543520+0000","last_became_peered":"2024-02-18T11:47:34.543520+0000","last_unstale":"2024-02-18T11:47:40.309074+0000","last_undegraded":"2024-02-18T11:47:40.309074+0000","last_fullsized":"2024-02-18T11:47:40.309074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554409+0000","last_change":"2024-02-18T11:47:33.519311+0000","last_active":"2024-02-18T11:47:39.554409+0000","last_peered":"2024-02-18T11:47:39.554409+0000","last_clean":"2024-02-18T11:47:39.554409+0000","last_became_active":"2024-02-18T11:47:33.518936+0000","last_became_peered":"2024-02-18T11:47:33.518936+0000","last_unstale":"2024-02-18T11:47:39.554409+0000","last_undegraded":"2024-02-18T11:47:39.554409+0000","last_fullsized":"2024-02-18T11:47:39.554409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624339+0000","last_change":"2024-02-18T11:47:38.568770+0000","last_active":"2024-02-18T11:47:39.624339+0000","last_peered":"2024-02-18T11:47:39.624339+0000","last_clean":"2024-02-18T11:47:39.624339+0000","last_became_active":"2024-02-18T11:47:38.568394+0000","last_became_peered":"2024-02-18T11:47:38.568394+0000","last_unstale":"2024-02-18T11:47:39.624339+0000","last_undegraded":"2024-02-18T11:47:39.624339+0000","last_fullsized":"2024-02-18T11:47:39.624339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:04.112912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556127+0000","last_change":"2024-02-18T11:47:34.530166+0000","last_active":"2024-02-18T11:47:39.556127+0000","last_peered":"2024-02-18T11:47:39.556127+0000","last_clean":"2024-02-18T11:47:39.556127+0000","last_became_active":"2024-02-18T11:47:34.529875+0000","last_became_peered":"2024-02-18T11:47:34.529875+0000","last_unstale":"2024-02-18T11:47:39.556127+0000","last_undegraded":"2024-02-18T11:47:39.556127+0000","last_fullsized":"2024-02-18T11:47:39.556127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556190+0000","last_change":"2024-02-18T11:47:33.519858+0000","last_active":"2024-02-18T11:47:39.556190+0000","last_peered":"2024-02-18T11:47:39.556190+0000","last_clean":"2024-02-18T11:47:39.556190+0000","last_became_active":"2024-02-18T11:47:33.519239+0000","last_became_peered":"2024-02-18T11:47:33.519239+0000","last_unstale":"2024-02-18T11:47:39.556190+0000","last_undegraded":"2024-02-18T11:47:39.556190+0000","last_fullsized":"2024-02-18T11:47:39.556190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555696+0000","last_change":"2024-02-18T11:47:38.583646+0000","last_active":"2024-02-18T11:47:39.555696+0000","last_peered":"2024-02-18T11:47:39.555696+0000","last_clean":"2024-02-18T11:47:39.555696+0000","last_became_active":"2024-02-18T11:47:38.583283+0000","last_became_peered":"2024-02-18T11:47:38.583283+0000","last_unstale":"2024-02-18T11:47:39.555696+0000","last_undegraded":"2024-02-18T11:47:39.555696+0000","last_fullsized":"2024-02-18T11:47:39.555696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309237+0000","last_change":"2024-02-18T11:47:34.549395+0000","last_active":"2024-02-18T11:47:40.309237+0000","last_peered":"2024-02-18T11:47:40.309237+0000","last_clean":"2024-02-18T11:47:40.309237+0000","last_became_active":"2024-02-18T11:47:34.549157+0000","last_became_peered":"2024-02-18T11:47:34.549157+0000","last_unstale":"2024-02-18T11:47:40.309237+0000","last_undegraded":"2024-02-18T11:47:40.309237+0000","last_fullsized":"2024-02-18T11:47:40.309237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172210+0000","last_change":"2024-02-18T11:47:33.526266+0000","last_active":"2024-02-18T11:47:40.172210+0000","last_peered":"2024-02-18T11:47:40.172210+0000","last_clean":"2024-02-18T11:47:40.172210+0000","last_became_active":"2024-02-18T11:47:33.525885+0000","last_became_peered":"2024-02-18T11:47:33.525885+0000","last_unstale":"2024-02-18T11:47:40.172210+0000","last_undegraded":"2024-02-18T11:47:40.172210+0000","last_fullsized":"2024-02-18T11:47:40.172210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555736+0000","last_change":"2024-02-18T11:47:38.581784+0000","last_active":"2024-02-18T11:47:39.555736+0000","last_peered":"2024-02-18T11:47:39.555736+0000","last_clean":"2024-02-18T11:47:39.555736+0000","last_became_active":"2024-02-18T11:47:38.581391+0000","last_became_peered":"2024-02-18T11:47:38.581391+0000","last_unstale":"2024-02-18T11:47:39.555736+0000","last_undegraded":"2024-02-18T11:47:39.555736+0000","last_fullsized":"2024-02-18T11:47:39.555736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624104+0000","last_change":"2024-02-18T11:47:34.537306+0000","last_active":"2024-02-18T11:47:39.624104+0000","last_peered":"2024-02-18T11:47:39.624104+0000","last_clean":"2024-02-18T11:47:39.624104+0000","last_became_active":"2024-02-18T11:47:34.537048+0000","last_became_peered":"2024-02-18T11:47:34.537048+0000","last_unstale":"2024-02-18T11:47:39.624104+0000","last_undegraded":"2024-02-18T11:47:39.624104+0000","last_fullsized":"2024-02-18T11:47:39.624104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:19:55.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624290+0000","last_change":"2024-02-18T11:47:33.518377+0000","last_active":"2024-02-18T11:47:39.624290+0000","last_peered":"2024-02-18T11:47:39.624290+0000","last_clean":"2024-02-18T11:47:39.624290+0000","last_became_active":"2024-02-18T11:47:33.518105+0000","last_became_peered":"2024-02-18T11:47:33.518105+0000","last_unstale":"2024-02-18T11:47:39.624290+0000","last_undegraded":"2024-02-18T11:47:39.624290+0000","last_fullsized":"2024-02-18T11:47:39.624290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:12:42.834778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172424+0000","last_change":"2024-02-18T11:47:38.585105+0000","last_active":"2024-02-18T11:47:40.172424+0000","last_peered":"2024-02-18T11:47:40.172424+0000","last_clean":"2024-02-18T11:47:40.172424+0000","last_became_active":"2024-02-18T11:47:38.584660+0000","last_became_peered":"2024-02-18T11:47:38.584660+0000","last_unstale":"2024-02-18T11:47:40.172424+0000","last_undegraded":"2024-02-18T11:47:40.172424+0000","last_fullsized":"2024-02-18T11:47:40.172424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309312+0000","last_change":"2024-02-18T11:47:34.546219+0000","last_active":"2024-02-18T11:47:40.309312+0000","last_peered":"2024-02-18T11:47:40.309312+0000","last_clean":"2024-02-18T11:47:40.309312+0000","last_became_active":"2024-02-18T11:47:34.545961+0000","last_became_peered":"2024-02-18T11:47:34.545961+0000","last_unstale":"2024-02-18T11:47:40.309312+0000","last_undegraded":"2024-02-18T11:47:40.309312+0000","last_fullsized":"2024-02-18T11:47:40.309312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:43.783562+0000","last_change":"2024-02-18T11:47:33.511834+0000","last_active":"2024-02-18T11:47:43.783562+0000","last_peered":"2024-02-18T11:47:43.783562+0000","last_clean":"2024-02-18T11:47:43.783562+0000","last_became_active":"2024-02-18T11:47:33.511537+0000","last_became_peered":"2024-02-18T11:47:33.511537+0000","last_unstale":"2024-02-18T11:47:43.783562+0000","last_undegraded":"2024-02-18T11:47:43.783562+0000","last_fullsized":"2024-02-18T11:47:43.783562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845016+0000","last_change":"2024-02-18T11:47:38.570461+0000","last_active":"2024-02-18T11:47:39.845016+0000","last_peered":"2024-02-18T11:47:39.845016+0000","last_clean":"2024-02-18T11:47:39.845016+0000","last_became_active":"2024-02-18T11:47:38.568615+0000","last_became_peered":"2024-02-18T11:47:38.568615+0000","last_unstale":"2024-02-18T11:47:39.845016+0000","last_undegraded":"2024-02-18T11:47:39.845016+0000","last_fullsized":"2024-02-18T11:47:39.845016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808848+0000","last_change":"2024-02-18T11:47:34.541278+0000","last_active":"2024-02-18T11:47:39.808848+0000","last_peered":"2024-02-18T11:47:39.808848+0000","last_clean":"2024-02-18T11:47:39.808848+0000","last_became_active":"2024-02-18T11:47:34.540891+0000","last_became_peered":"2024-02-18T11:47:34.540891+0000","last_unstale":"2024-02-18T11:47:39.808848+0000","last_undegraded":"2024-02-18T11:47:39.808848+0000","last_fullsized":"2024-02-18T11:47:39.808848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845415+0000","last_change":"2024-02-18T11:47:33.526021+0000","last_active":"2024-02-18T11:47:39.845415+0000","last_peered":"2024-02-18T11:47:39.845415+0000","last_clean":"2024-02-18T11:47:39.845415+0000","last_became_active":"2024-02-18T11:47:33.525810+0000","last_became_peered":"2024-02-18T11:47:33.525810+0000","last_unstale":"2024-02-18T11:47:39.845415+0000","last_undegraded":"2024-02-18T11:47:39.845415+0000","last_fullsized":"2024-02-18T11:47:39.845415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809063+0000","last_change":"2024-02-18T11:47:38.568064+0000","last_active":"2024-02-18T11:47:39.809063+0000","last_peered":"2024-02-18T11:47:39.809063+0000","last_clean":"2024-02-18T11:47:39.809063+0000","last_became_active":"2024-02-18T11:47:38.567648+0000","last_became_peered":"2024-02-18T11:47:38.567648+0000","last_unstale":"2024-02-18T11:47:39.809063+0000","last_undegraded":"2024-02-18T11:47:39.809063+0000","last_fullsized":"2024-02-18T11:47:39.809063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:41.335475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624622+0000","last_change":"2024-02-18T11:47:34.546545+0000","last_active":"2024-02-18T11:47:39.624622+0000","last_peered":"2024-02-18T11:47:39.624622+0000","last_clean":"2024-02-18T11:47:39.624622+0000","last_became_active":"2024-02-18T11:47:34.545918+0000","last_became_peered":"2024-02-18T11:47:34.545918+0000","last_unstale":"2024-02-18T11:47:39.624622+0000","last_undegraded":"2024-02-18T11:47:39.624622+0000","last_fullsized":"2024-02-18T11:47:39.624622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172369+0000","last_change":"2024-02-18T11:47:33.526119+0000","last_active":"2024-02-18T11:47:40.172369+0000","last_peered":"2024-02-18T11:47:40.172369+0000","last_clean":"2024-02-18T11:47:40.172369+0000","last_became_active":"2024-02-18T11:47:33.525722+0000","last_became_peered":"2024-02-18T11:47:33.525722+0000","last_unstale":"2024-02-18T11:47:40.172369+0000","last_undegraded":"2024-02-18T11:47:40.172369+0000","last_fullsized":"2024-02-18T11:47:40.172369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555321+0000","last_change":"2024-02-18T11:47:38.569853+0000","last_active":"2024-02-18T11:47:39.555321+0000","last_peered":"2024-02-18T11:47:39.555321+0000","last_clean":"2024-02-18T11:47:39.555321+0000","last_became_active":"2024-02-18T11:47:38.569315+0000","last_became_peered":"2024-02-18T11:47:38.569315+0000","last_unstale":"2024-02-18T11:47:39.555321+0000","last_undegraded":"2024-02-18T11:47:39.555321+0000","last_fullsized":"2024-02-18T11:47:39.555321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:22:24.799952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172285+0000","last_change":"2024-02-18T11:47:34.538399+0000","last_active":"2024-02-18T11:47:40.172285+0000","last_peered":"2024-02-18T11:47:40.172285+0000","last_clean":"2024-02-18T11:47:40.172285+0000","last_became_active":"2024-02-18T11:47:34.538133+0000","last_became_peered":"2024-02-18T11:47:34.538133+0000","last_unstale":"2024-02-18T11:47:40.172285+0000","last_undegraded":"2024-02-18T11:47:40.172285+0000","last_fullsized":"2024-02-18T11:47:40.172285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556798+0000","last_change":"2024-02-18T11:47:33.526252+0000","last_active":"2024-02-18T11:47:39.556798+0000","last_peered":"2024-02-18T11:47:39.556798+0000","last_clean":"2024-02-18T11:47:39.556798+0000","last_became_active":"2024-02-18T11:47:33.525952+0000","last_became_peered":"2024-02-18T11:47:33.525952+0000","last_unstale":"2024-02-18T11:47:39.556798+0000","last_undegraded":"2024-02-18T11:47:39.556798+0000","last_fullsized":"2024-02-18T11:47:39.556798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555129+0000","last_change":"2024-02-18T11:47:38.583952+0000","last_active":"2024-02-18T11:47:39.555129+0000","last_peered":"2024-02-18T11:47:39.555129+0000","last_clean":"2024-02-18T11:47:39.555129+0000","last_became_active":"2024-02-18T11:47:38.583579+0000","last_became_peered":"2024-02-18T11:47:38.583579+0000","last_unstale":"2024-02-18T11:47:39.555129+0000","last_undegraded":"2024-02-18T11:47:39.555129+0000","last_fullsized":"2024-02-18T11:47:39.555129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:33:29.418521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624283+0000","last_change":"2024-02-18T11:47:38.583643+0000","last_active":"2024-02-18T11:47:39.624283+0000","last_peered":"2024-02-18T11:47:39.624283+0000","last_clean":"2024-02-18T11:47:39.624283+0000","last_became_active":"2024-02-18T11:47:38.583245+0000","last_became_peered":"2024-02-18T11:47:38.583245+0000","last_unstale":"2024-02-18T11:47:39.624283+0000","last_undegraded":"2024-02-18T11:47:39.624283+0000","last_fullsized":"2024-02-18T11:47:39.624283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844764+0000","last_change":"2024-02-18T11:47:34.540661+0000","last_active":"2024-02-18T11:47:39.844764+0000","last_peered":"2024-02-18T11:47:39.844764+0000","last_clean":"2024-02-18T11:47:39.844764+0000","last_became_active":"2024-02-18T11:47:34.540342+0000","last_became_peered":"2024-02-18T11:47:34.540342+0000","last_unstale":"2024-02-18T11:47:39.844764+0000","last_undegraded":"2024-02-18T11:47:39.844764+0000","last_fullsized":"2024-02-18T11:47:39.844764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171940+0000","last_change":"2024-02-18T11:47:33.526132+0000","last_active":"2024-02-18T11:47:40.171940+0000","last_peered":"2024-02-18T11:47:40.171940+0000","last_clean":"2024-02-18T11:47:40.171940+0000","last_became_active":"2024-02-18T11:47:33.525798+0000","last_became_peered":"2024-02-18T11:47:33.525798+0000","last_unstale":"2024-02-18T11:47:40.171940+0000","last_undegraded":"2024-02-18T11:47:40.171940+0000","last_fullsized":"2024-02-18T11:47:40.171940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556839+0000","last_change":"2024-02-18T11:47:38.572273+0000","last_active":"2024-02-18T11:47:39.556839+0000","last_peered":"2024-02-18T11:47:39.556839+0000","last_clean":"2024-02-18T11:47:39.556839+0000","last_became_active":"2024-02-18T11:47:38.571852+0000","last_became_peered":"2024-02-18T11:47:38.571852+0000","last_unstale":"2024-02-18T11:47:39.556839+0000","last_undegraded":"2024-02-18T11:47:39.556839+0000","last_fullsized":"2024-02-18T11:47:39.556839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555386+0000","last_change":"2024-02-18T11:47:34.546984+0000","last_active":"2024-02-18T11:47:39.555386+0000","last_peered":"2024-02-18T11:47:39.555386+0000","last_clean":"2024-02-18T11:47:39.555386+0000","last_became_active":"2024-02-18T11:47:34.546738+0000","last_became_peered":"2024-02-18T11:47:34.546738+0000","last_unstale":"2024-02-18T11:47:39.555386+0000","last_undegraded":"2024-02-18T11:47:39.555386+0000","last_fullsized":"2024-02-18T11:47:39.555386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555662+0000","last_change":"2024-02-18T11:47:33.514025+0000","last_active":"2024-02-18T11:47:39.555662+0000","last_peered":"2024-02-18T11:47:39.555662+0000","last_clean":"2024-02-18T11:47:39.555662+0000","last_became_active":"2024-02-18T11:47:33.513762+0000","last_became_peered":"2024-02-18T11:47:33.513762+0000","last_unstale":"2024-02-18T11:47:39.555662+0000","last_undegraded":"2024-02-18T11:47:39.555662+0000","last_fullsized":"2024-02-18T11:47:39.555662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556529+0000","last_change":"2024-02-18T11:47:33.521856+0000","last_active":"2024-02-18T11:47:39.556529+0000","last_peered":"2024-02-18T11:47:39.556529+0000","last_clean":"2024-02-18T11:47:39.556529+0000","last_became_active":"2024-02-18T11:47:33.521563+0000","last_became_peered":"2024-02-18T11:47:33.521563+0000","last_unstale":"2024-02-18T11:47:39.556529+0000","last_undegraded":"2024-02-18T11:47:39.556529+0000","last_fullsized":"2024-02-18T11:47:39.556529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624276+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.624276+0000","last_peered":"2024-02-18T11:47:39.624276+0000","last_clean":"2024-02-18T11:47:39.624276+0000","last_became_active":"2024-02-18T11:47:34.536344+0000","last_became_peered":"2024-02-18T11:47:34.536344+0000","last_unstale":"2024-02-18T11:47:39.624276+0000","last_undegraded":"2024-02-18T11:47:39.624276+0000","last_fullsized":"2024-02-18T11:47:39.624276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554790+0000","last_change":"2024-02-18T11:47:38.572780+0000","last_active":"2024-02-18T11:47:39.554790+0000","last_peered":"2024-02-18T11:47:39.554790+0000","last_clean":"2024-02-18T11:47:39.554790+0000","last_became_active":"2024-02-18T11:47:38.572401+0000","last_became_peered":"2024-02-18T11:47:38.572401+0000","last_unstale":"2024-02-18T11:47:39.554790+0000","last_undegraded":"2024-02-18T11:47:39.554790+0000","last_fullsized":"2024-02-18T11:47:39.554790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:06.469768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555607+0000","last_change":"2024-02-18T11:47:33.521346+0000","last_active":"2024-02-18T11:47:39.555607+0000","last_peered":"2024-02-18T11:47:39.555607+0000","last_clean":"2024-02-18T11:47:39.555607+0000","last_became_active":"2024-02-18T11:47:33.521077+0000","last_became_peered":"2024-02-18T11:47:33.521077+0000","last_unstale":"2024-02-18T11:47:39.555607+0000","last_undegraded":"2024-02-18T11:47:39.555607+0000","last_fullsized":"2024-02-18T11:47:39.555607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808655+0000","last_change":"2024-02-18T11:47:34.535139+0000","last_active":"2024-02-18T11:47:39.808655+0000","last_peered":"2024-02-18T11:47:39.808655+0000","last_clean":"2024-02-18T11:47:39.808655+0000","last_became_active":"2024-02-18T11:47:34.534871+0000","last_became_peered":"2024-02-18T11:47:34.534871+0000","last_unstale":"2024-02-18T11:47:39.808655+0000","last_undegraded":"2024-02-18T11:47:39.808655+0000","last_fullsized":"2024-02-18T11:47:39.808655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:58.805082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556009+0000","last_change":"2024-02-18T11:47:38.566494+0000","last_active":"2024-02-18T11:47:39.556009+0000","last_peered":"2024-02-18T11:47:39.556009+0000","last_clean":"2024-02-18T11:47:39.556009+0000","last_became_active":"2024-02-18T11:47:38.565672+0000","last_became_peered":"2024-02-18T11:47:38.565672+0000","last_unstale":"2024-02-18T11:47:39.556009+0000","last_undegraded":"2024-02-18T11:47:39.556009+0000","last_fullsized":"2024-02-18T11:47:39.556009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:38:29.952609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556029+0000","last_change":"2024-02-18T11:47:34.531011+0000","last_active":"2024-02-18T11:47:39.556029+0000","last_peered":"2024-02-18T11:47:39.556029+0000","last_clean":"2024-02-18T11:47:39.556029+0000","last_became_active":"2024-02-18T11:47:34.530758+0000","last_became_peered":"2024-02-18T11:47:34.530758+0000","last_unstale":"2024-02-18T11:47:39.556029+0000","last_undegraded":"2024-02-18T11:47:39.556029+0000","last_fullsized":"2024-02-18T11:47:39.556029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554518+0000","last_change":"2024-02-18T11:47:34.543755+0000","last_active":"2024-02-18T11:47:39.554518+0000","last_peered":"2024-02-18T11:47:39.554518+0000","last_clean":"2024-02-18T11:47:39.554518+0000","last_became_active":"2024-02-18T11:47:34.543500+0000","last_became_peered":"2024-02-18T11:47:34.543500+0000","last_unstale":"2024-02-18T11:47:39.554518+0000","last_undegraded":"2024-02-18T11:47:39.554518+0000","last_fullsized":"2024-02-18T11:47:39.554518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172079+0000","last_change":"2024-02-18T11:47:34.545316+0000","last_active":"2024-02-18T11:47:40.172079+0000","last_peered":"2024-02-18T11:47:40.172079+0000","last_clean":"2024-02-18T11:47:40.172079+0000","last_became_active":"2024-02-18T11:47:34.545018+0000","last_became_peered":"2024-02-18T11:47:34.545018+0000","last_unstale":"2024-02-18T11:47:40.172079+0000","last_undegraded":"2024-02-18T11:47:40.172079+0000","last_fullsized":"2024-02-18T11:47:40.172079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624183+0000","last_change":"2024-02-18T11:47:34.546010+0000","last_active":"2024-02-18T11:47:39.624183+0000","last_peered":"2024-02-18T11:47:39.624183+0000","last_clean":"2024-02-18T11:47:39.624183+0000","last_became_active":"2024-02-18T11:47:34.545247+0000","last_became_peered":"2024-02-18T11:47:34.545247+0000","last_unstale":"2024-02-18T11:47:39.624183+0000","last_undegraded":"2024-02-18T11:47:39.624183+0000","last_fullsized":"2024-02-18T11:47:39.624183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555993+0000","last_change":"2024-02-18T11:47:34.538436+0000","last_active":"2024-02-18T11:47:39.555993+0000","last_peered":"2024-02-18T11:47:39.555993+0000","last_clean":"2024-02-18T11:47:39.555993+0000","last_became_active":"2024-02-18T11:47:34.538207+0000","last_became_peered":"2024-02-18T11:47:34.538207+0000","last_unstale":"2024-02-18T11:47:39.555993+0000","last_undegraded":"2024-02-18T11:47:39.555993+0000","last_fullsized":"2024-02-18T11:47:39.555993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845187+0000","last_change":"2024-02-18T11:47:34.542741+0000","last_active":"2024-02-18T11:47:39.845187+0000","last_peered":"2024-02-18T11:47:39.845187+0000","last_clean":"2024-02-18T11:47:39.845187+0000","last_became_active":"2024-02-18T11:47:34.542467+0000","last_became_peered":"2024-02-18T11:47:34.542467+0000","last_unstale":"2024-02-18T11:47:39.845187+0000","last_undegraded":"2024-02-18T11:47:39.845187+0000","last_fullsized":"2024-02-18T11:47:39.845187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557105+0000","last_change":"2024-02-18T11:47:34.539970+0000","last_active":"2024-02-18T11:47:39.557105+0000","last_peered":"2024-02-18T11:47:39.557105+0000","last_clean":"2024-02-18T11:47:39.557105+0000","last_became_active":"2024-02-18T11:47:34.539471+0000","last_became_peered":"2024-02-18T11:47:34.539471+0000","last_unstale":"2024-02-18T11:47:39.557105+0000","last_undegraded":"2024-02-18T11:47:39.557105+0000","last_fullsized":"2024-02-18T11:47:39.557105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555437+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.555437+0000","last_peered":"2024-02-18T11:47:39.555437+0000","last_clean":"2024-02-18T11:47:39.555437+0000","last_became_active":"2024-02-18T11:47:34.536356+0000","last_became_peered":"2024-02-18T11:47:34.536356+0000","last_unstale":"2024-02-18T11:47:39.555437+0000","last_undegraded":"2024-02-18T11:47:39.555437+0000","last_fullsized":"2024-02-18T11:47:39.555437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171840+0000","last_change":"2024-02-18T11:47:34.542717+0000","last_active":"2024-02-18T11:47:40.171840+0000","last_peered":"2024-02-18T11:47:40.171840+0000","last_clean":"2024-02-18T11:47:40.171840+0000","last_became_active":"2024-02-18T11:47:34.542470+0000","last_became_peered":"2024-02-18T11:47:34.542470+0000","last_unstale":"2024-02-18T11:47:40.171840+0000","last_undegraded":"2024-02-18T11:47:40.171840+0000","last_fullsized":"2024-02-18T11:47:40.171840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555193+0000","last_change":"2024-02-18T11:47:34.537727+0000","last_active":"2024-02-18T11:47:39.555193+0000","last_peered":"2024-02-18T11:47:39.555193+0000","last_clean":"2024-02-18T11:47:39.555193+0000","last_became_active":"2024-02-18T11:47:34.536902+0000","last_became_peered":"2024-02-18T11:47:34.536902+0000","last_unstale":"2024-02-18T11:47:39.555193+0000","last_undegraded":"2024-02-18T11:47:39.555193+0000","last_fullsized":"2024-02-18T11:47:39.555193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172026+0000","last_change":"2024-02-18T11:47:34.549313+0000","last_active":"2024-02-18T11:47:40.172026+0000","last_peered":"2024-02-18T11:47:40.172026+0000","last_clean":"2024-02-18T11:47:40.172026+0000","last_became_active":"2024-02-18T11:47:34.549010+0000","last_became_peered":"2024-02-18T11:47:34.549010+0000","last_unstale":"2024-02-18T11:47:40.172026+0000","last_undegraded":"2024-02-18T11:47:40.172026+0000","last_fullsized":"2024-02-18T11:47:40.172026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624061+0000","last_change":"2024-02-18T11:47:34.546238+0000","last_active":"2024-02-18T11:47:39.624061+0000","last_peered":"2024-02-18T11:47:39.624061+0000","last_clean":"2024-02-18T11:47:39.624061+0000","last_became_active":"2024-02-18T11:47:34.545634+0000","last_became_peered":"2024-02-18T11:47:34.545634+0000","last_unstale":"2024-02-18T11:47:39.624061+0000","last_undegraded":"2024-02-18T11:47:39.624061+0000","last_fullsized":"2024-02-18T11:47:39.624061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:18:21.067312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556482+0000","last_change":"2024-02-18T11:47:34.534576+0000","last_active":"2024-02-18T11:47:39.556482+0000","last_peered":"2024-02-18T11:47:39.556482+0000","last_clean":"2024-02-18T11:47:39.556482+0000","last_became_active":"2024-02-18T11:47:34.534232+0000","last_became_peered":"2024-02-18T11:47:34.534232+0000","last_unstale":"2024-02-18T11:47:39.556482+0000","last_undegraded":"2024-02-18T11:47:39.556482+0000","last_fullsized":"2024-02-18T11:47:39.556482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309588+0000","last_change":"2024-02-18T11:47:34.546750+0000","last_active":"2024-02-18T11:47:40.309588+0000","last_peered":"2024-02-18T11:47:40.309588+0000","last_clean":"2024-02-18T11:47:40.309588+0000","last_became_active":"2024-02-18T11:47:34.546507+0000","last_became_peered":"2024-02-18T11:47:34.546507+0000","last_unstale":"2024-02-18T11:47:40.309588+0000","last_undegraded":"2024-02-18T11:47:40.309588+0000","last_fullsized":"2024-02-18T11:47:40.309588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555043+0000","last_change":"2024-02-18T11:47:34.542028+0000","last_active":"2024-02-18T11:47:39.555043+0000","last_peered":"2024-02-18T11:47:39.555043+0000","last_clean":"2024-02-18T11:47:39.555043+0000","last_became_active":"2024-02-18T11:47:34.541653+0000","last_became_peered":"2024-02-18T11:47:34.541653+0000","last_unstale":"2024-02-18T11:47:39.555043+0000","last_undegraded":"2024-02-18T11:47:39.555043+0000","last_fullsized":"2024-02-18T11:47:39.555043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:38.652128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172499+0000","last_change":"2024-02-18T11:47:34.549989+0000","last_active":"2024-02-18T11:47:40.172499+0000","last_peered":"2024-02-18T11:47:40.172499+0000","last_clean":"2024-02-18T11:47:40.172499+0000","last_became_active":"2024-02-18T11:47:34.549661+0000","last_became_peered":"2024-02-18T11:47:34.549661+0000","last_unstale":"2024-02-18T11:47:40.172499+0000","last_undegraded":"2024-02-18T11:47:40.172499+0000","last_fullsized":"2024-02-18T11:47:40.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171732+0000","last_change":"2024-02-18T11:47:34.547081+0000","last_active":"2024-02-18T11:47:40.171732+0000","last_peered":"2024-02-18T11:47:40.171732+0000","last_clean":"2024-02-18T11:47:40.171732+0000","last_became_active":"2024-02-18T11:47:34.546812+0000","last_became_peered":"2024-02-18T11:47:34.546812+0000","last_unstale":"2024-02-18T11:47:40.171732+0000","last_undegraded":"2024-02-18T11:47:40.171732+0000","last_fullsized":"2024-02-18T11:47:40.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844848+0000","last_change":"2024-02-18T11:47:34.534907+0000","last_active":"2024-02-18T11:47:39.844848+0000","last_peered":"2024-02-18T11:47:39.844848+0000","last_clean":"2024-02-18T11:47:39.844848+0000","last_became_active":"2024-02-18T11:47:34.534661+0000","last_became_peered":"2024-02-18T11:47:34.534661+0000","last_unstale":"2024-02-18T11:47:39.844848+0000","last_undegraded":"2024-02-18T11:47:39.844848+0000","last_fullsized":"2024-02-18T11:47:39.844848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624050+0000","last_change":"2024-02-18T11:47:34.537888+0000","last_active":"2024-02-18T11:47:39.624050+0000","last_peered":"2024-02-18T11:47:39.624050+0000","last_clean":"2024-02-18T11:47:39.624050+0000","last_became_active":"2024-02-18T11:47:34.537647+0000","last_became_peered":"2024-02-18T11:47:34.537647+0000","last_unstale":"2024-02-18T11:47:39.624050+0000","last_undegraded":"2024-02-18T11:47:39.624050+0000","last_fullsized":"2024-02-18T11:47:39.624050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:02.426785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555401+0000","last_change":"2024-02-18T11:47:34.541254+0000","last_active":"2024-02-18T11:47:39.555401+0000","last_peered":"2024-02-18T11:47:39.555401+0000","last_clean":"2024-02-18T11:47:39.555401+0000","last_became_active":"2024-02-18T11:47:34.540955+0000","last_became_peered":"2024-02-18T11:47:34.540955+0000","last_unstale":"2024-02-18T11:47:39.555401+0000","last_undegraded":"2024-02-18T11:47:39.555401+0000","last_fullsized":"2024-02-18T11:47:39.555401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808837+0000","last_change":"2024-02-18T11:47:34.531217+0000","last_active":"2024-02-18T11:47:39.808837+0000","last_peered":"2024-02-18T11:47:39.808837+0000","last_clean":"2024-02-18T11:47:39.808837+0000","last_became_active":"2024-02-18T11:47:34.530120+0000","last_became_peered":"2024-02-18T11:47:34.530120+0000","last_unstale":"2024-02-18T11:47:39.808837+0000","last_undegraded":"2024-02-18T11:47:39.808837+0000","last_fullsized":"2024-02-18T11:47:39.808837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:43.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556645+0000","last_change":"2024-02-18T11:47:34.544075+0000","last_active":"2024-02-18T11:47:39.556645+0000","last_peered":"2024-02-18T11:47:39.556645+0000","last_clean":"2024-02-18T11:47:39.556645+0000","last_became_active":"2024-02-18T11:47:34.543761+0000","last_became_peered":"2024-02-18T11:47:34.543761+0000","last_unstale":"2024-02-18T11:47:39.556645+0000","last_undegraded":"2024-02-18T11:47:39.556645+0000","last_fullsized":"2024-02-18T11:47:39.556645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557031+0000","last_change":"2024-02-18T11:47:34.546387+0000","last_active":"2024-02-18T11:47:39.557031+0000","last_peered":"2024-02-18T11:47:39.557031+0000","last_clean":"2024-02-18T11:47:39.557031+0000","last_became_active":"2024-02-18T11:47:34.546115+0000","last_became_peered":"2024-02-18T11:47:34.546115+0000","last_unstale":"2024-02-18T11:47:39.557031+0000","last_undegraded":"2024-02-18T11:47:39.557031+0000","last_fullsized":"2024-02-18T11:47:39.557031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309697+0000","last_change":"2024-02-18T11:47:34.552880+0000","last_active":"2024-02-18T11:47:40.309697+0000","last_peered":"2024-02-18T11:47:40.309697+0000","last_clean":"2024-02-18T11:47:40.309697+0000","last_became_active":"2024-02-18T11:47:34.552530+0000","last_became_peered":"2024-02-18T11:47:34.552530+0000","last_unstale":"2024-02-18T11:47:40.309697+0000","last_undegraded":"2024-02-18T11:47:40.309697+0000","last_fullsized":"2024-02-18T11:47:40.309697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555106+0000","last_change":"2024-02-18T11:47:34.545727+0000","last_active":"2024-02-18T11:47:39.555106+0000","last_peered":"2024-02-18T11:47:39.555106+0000","last_clean":"2024-02-18T11:47:39.555106+0000","last_became_active":"2024-02-18T11:47:34.545475+0000","last_became_peered":"2024-02-18T11:47:34.545475+0000","last_unstale":"2024-02-18T11:47:39.555106+0000","last_undegraded":"2024-02-18T11:47:39.555106+0000","last_fullsized":"2024-02-18T11:47:39.555106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309012+0000","last_change":"2024-02-18T11:47:34.547696+0000","last_active":"2024-02-18T11:47:40.309012+0000","last_peered":"2024-02-18T11:47:40.309012+0000","last_clean":"2024-02-18T11:47:40.309012+0000","last_became_active":"2024-02-18T11:47:34.547448+0000","last_became_peered":"2024-02-18T11:47:34.547448+0000","last_unstale":"2024-02-18T11:47:40.309012+0000","last_undegraded":"2024-02-18T11:47:40.309012+0000","last_fullsized":"2024-02-18T11:47:40.309012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171835+0000","last_change":"2024-02-18T11:47:34.548053+0000","last_active":"2024-02-18T11:47:40.171835+0000","last_peered":"2024-02-18T11:47:40.171835+0000","last_clean":"2024-02-18T11:47:40.171835+0000","last_became_active":"2024-02-18T11:47:34.547181+0000","last_became_peered":"2024-02-18T11:47:34.547181+0000","last_unstale":"2024-02-18T11:47:40.171835+0000","last_undegraded":"2024-02-18T11:47:40.171835+0000","last_fullsized":"2024-02-18T11:47:40.171835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309134+0000","last_change":"2024-02-18T11:47:34.549242+0000","last_active":"2024-02-18T11:47:40.309134+0000","last_peered":"2024-02-18T11:47:40.309134+0000","last_clean":"2024-02-18T11:47:40.309134+0000","last_became_active":"2024-02-18T11:47:34.548872+0000","last_became_peered":"2024-02-18T11:47:34.548872+0000","last_unstale":"2024-02-18T11:47:40.309134+0000","last_undegraded":"2024-02-18T11:47:40.309134+0000","last_fullsized":"2024-02-18T11:47:40.309134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172083+0000","last_change":"2024-02-18T11:47:34.546466+0000","last_active":"2024-02-18T11:47:40.172083+0000","last_peered":"2024-02-18T11:47:40.172083+0000","last_clean":"2024-02-18T11:47:40.172083+0000","last_became_active":"2024-02-18T11:47:34.546218+0000","last_became_peered":"2024-02-18T11:47:34.546218+0000","last_unstale":"2024-02-18T11:47:40.172083+0000","last_undegraded":"2024-02-18T11:47:40.172083+0000","last_fullsized":"2024-02-18T11:47:40.172083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624360+0000","last_change":"2024-02-18T11:47:34.530060+0000","last_active":"2024-02-18T11:47:39.624360+0000","last_peered":"2024-02-18T11:47:39.624360+0000","last_clean":"2024-02-18T11:47:39.624360+0000","last_became_active":"2024-02-18T11:47:34.529799+0000","last_became_peered":"2024-02-18T11:47:34.529799+0000","last_unstale":"2024-02-18T11:47:39.624360+0000","last_undegraded":"2024-02-18T11:47:39.624360+0000","last_fullsized":"2024-02-18T11:47:39.624360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845225+0000","last_change":"2024-02-18T11:47:34.547273+0000","last_active":"2024-02-18T11:47:39.845225+0000","last_peered":"2024-02-18T11:47:39.845225+0000","last_clean":"2024-02-18T11:47:39.845225+0000","last_became_active":"2024-02-18T11:47:34.546984+0000","last_became_peered":"2024-02-18T11:47:34.546984+0000","last_unstale":"2024-02-18T11:47:39.845225+0000","last_undegraded":"2024-02-18T11:47:39.845225+0000","last_fullsized":"2024-02-18T11:47:39.845225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309602+0000","last_change":"2024-02-18T11:47:34.547456+0000","last_active":"2024-02-18T11:47:40.309602+0000","last_peered":"2024-02-18T11:47:40.309602+0000","last_clean":"2024-02-18T11:47:40.309602+0000","last_became_active":"2024-02-18T11:47:34.547193+0000","last_became_peered":"2024-02-18T11:47:34.547193+0000","last_unstale":"2024-02-18T11:47:40.309602+0000","last_undegraded":"2024-02-18T11:47:40.309602+0000","last_fullsized":"2024-02-18T11:47:40.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:01.871 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:02.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.013+0000 7fe134325640 1 Processor -- start 2024-02-18T11:48:02.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.013+0000 7fe134325640 1 -- start start 2024-02-18T11:48:02.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe134325640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe12c06ea20 0x7fe12c1653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe134325640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c0b02e0 con 0x7fe12c15a860 2024-02-18T11:48:02.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe134325640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12c0af6d0 con 0x7fe12c15ac40 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe134325640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12c0b1780 con 0x7fe12c06ea20 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe1328c2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe12c06ea20 0x7fe12c1653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe1328c2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe12c06ea20 0x7fe12c1653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57572/0 (socket says 172.21.15.116:57572) 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe1328c2640 1 -- 172.21.15.116:0/1937319539 learned_addr learned my addr 172.21.15.116:0/1937319539 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe1328c2640 1 -- 172.21.15.116:0/1937319539 >> v1:172.21.15.143:6789/0 conn(0x7fe12c15ac40 legacy=0x7fe12c06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe1328c2640 1 -- 172.21.15.116:0/1937319539 >> v1:172.21.15.116:6789/0 conn(0x7fe12c15a860 legacy=0x7fe12c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.008 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.014+0000 7fe1328c2640 1 -- 172.21.15.116:0/1937319539 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c0af9b0 con 0x7fe12c06ea20 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe1328c2640 1 --2- 172.21.15.116:0/1937319539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c06ea20 0x7fe12c1653a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe12400d150 tx=0x7fe12402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e4abb7a5fc52c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe1310bf640 1 -- 172.21.15.116:0/1937319539 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400b0d0 con 0x7fe12c06ea20 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe1310bf640 1 -- 172.21.15.116:0/1937319539 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12400b230 con 0x7fe12c06ea20 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe1310bf640 1 -- 172.21.15.116:0/1937319539 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124009600 con 0x7fe12c06ea20 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 -- 172.21.15.116:0/1937319539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c06ea20 msgr2=0x7fe12c1653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 --2- 172.21.15.116:0/1937319539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c06ea20 0x7fe12c1653a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe12400d150 tx=0x7fe12402e170 comp rx=0 tx=0).stop 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 -- 172.21.15.116:0/1937319539 shutdown_connections 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 --2- 172.21.15.116:0/1937319539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c06ea20 0x7fe12c1653a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 -- 172.21.15.116:0/1937319539 >> 172.21.15.116:0/1937319539 conn(0x7fe12c066f20 msgr2=0x7fe12c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 -- 172.21.15.116:0/1937319539 shutdown_connections 2024-02-18T11:48:02.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.015+0000 7fe134325640 1 -- 172.21.15.116:0/1937319539 wait complete. 2024-02-18T11:48:02.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 Processor -- start 2024-02-18T11:48:02.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 -- start start 2024-02-18T11:48:02.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12c06ea20 0x7fe12c0a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe12c15a860 0x7fe12c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 0x7fe12c151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c157f60 con 0x7fe12c15ac40 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c156eb0 con 0x7fe12c06ea20 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe134325640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12c157190 con 0x7fe12c15a860 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe1328c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 0x7fe12c151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe1328c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 0x7fe12c151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57578/0 (socket says 172.21.15.116:57578) 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.017+0000 7fe1328c2640 1 -- 172.21.15.116:0/4186771919 learned_addr learned my addr 172.21.15.116:0/4186771919 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1318c0640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe12c15a860 0x7fe12c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1328c2640 1 -- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe12c15a860 msgr2=0x7fe12c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.014 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1320c1640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12c06ea20 0x7fe12c0a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1328c2640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe12c15a860 0x7fe12c156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1328c2640 1 -- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12c06ea20 msgr2=0x7fe12c0a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1328c2640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12c06ea20 0x7fe12c0a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1328c2640 1 -- 172.21.15.116:0/4186771919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12400bd40 con 0x7fe12c15ac40 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1318c0640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe12c15a860 0x7fe12c156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1320c1640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12c06ea20 0x7fe12c0a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe1328c2640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 0x7fe12c151520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe124038040 tx=0x7fe124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400b720 con 0x7fe12c15ac40 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe124040660 con 0x7fe12c15ac40 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12c157470 con 0x7fe12c15ac40 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1240409f0 con 0x7fe12c15ac40 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.018+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12c156bc0 con 0x7fe12c15ac40 2024-02-18T11:48:02.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.019+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12c157660 con 0x7fe12c15ac40 2024-02-18T11:48:02.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.019+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe124040b50 con 0x7fe12c15ac40 2024-02-18T11:48:02.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.020+0000 7fe122ffd640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe11006bc70 0x7fe11006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.020+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe1240afc10 con 0x7fe12c15ac40 2024-02-18T11:48:02.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.020+0000 7fe1320c1640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe11006bc70 0x7fe11006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.016 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.020+0000 7fe1320c1640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe11006bc70 0x7fe11006e130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe128001680 tx=0x7fe128009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.017 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.024+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe12407a0b0 con 0x7fe12c15ac40 2024-02-18T11:48:02.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.192+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe12c0cd3c0 con 0x7fe11006bc70 2024-02-18T11:48:02.192 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.199+0000 7fe122ffd640 1 -- 172.21.15.116:0/4186771919 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7fe128001c60 con 0x7fe11006bc70 2024-02-18T11:48:02.198 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:02.200 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe11006bc70 msgr2=0x7fe11006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe11006bc70 0x7fe11006e130 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe128001680 tx=0x7fe128009040 comp rx=0 tx=0).stop 2024-02-18T11:48:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 msgr2=0x7fe12c151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 0x7fe12c151520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe124038040 tx=0x7fe124004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 shutdown_connections 2024-02-18T11:48:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe11006bc70 0x7fe11006e130 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe12c15ac40 0x7fe12c151520 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe12c15a860 0x7fe12c156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 --2- 172.21.15.116:0/4186771919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12c06ea20 0x7fe12c0a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 >> 172.21.15.116:0/4186771919 conn(0x7fe12c066f20 msgr2=0x7fe12c0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 shutdown_connections 2024-02-18T11:48:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.203+0000 7fe134325640 1 -- 172.21.15.116:0/4186771919 wait complete. 2024-02-18T11:48:02.207 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-02-18T11:48:00.295774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219380,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755340,"statfs":{"total":773094113280,"available":772869468160,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003251"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624689+0000","last_change":"2024-02-18T11:47:33.523482+0000","last_active":"2024-02-18T11:47:39.624689+0000","last_peered":"2024-02-18T11:47:39.624689+0000","last_clean":"2024-02-18T11:47:39.624689+0000","last_became_active":"2024-02-18T11:47:33.523129+0000","last_became_peered":"2024-02-18T11:47:33.523129+0000","last_unstale":"2024-02-18T11:47:39.624689+0000","last_undegraded":"2024-02-18T11:47:39.624689+0000","last_fullsized":"2024-02-18T11:47:39.624689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555924+0000","last_change":"2024-02-18T11:47:34.538612+0000","last_active":"2024-02-18T11:47:39.555924+0000","last_peered":"2024-02-18T11:47:39.555924+0000","last_clean":"2024-02-18T11:47:39.555924+0000","last_became_active":"2024-02-18T11:47:34.538315+0000","last_became_peered":"2024-02-18T11:47:34.538315+0000","last_unstale":"2024-02-18T11:47:39.555924+0000","last_undegraded":"2024-02-18T11:47:39.555924+0000","last_fullsized":"2024-02-18T11:47:39.555924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555044+0000","last_change":"2024-02-18T11:47:38.585025+0000","last_active":"2024-02-18T11:47:39.555044+0000","last_peered":"2024-02-18T11:47:39.555044+0000","last_clean":"2024-02-18T11:47:39.555044+0000","last_became_active":"2024-02-18T11:47:38.584669+0000","last_became_peered":"2024-02-18T11:47:38.584669+0000","last_unstale":"2024-02-18T11:47:39.555044+0000","last_undegraded":"2024-02-18T11:47:39.555044+0000","last_fullsized":"2024-02-18T11:47:39.555044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:07:55.638374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556025+0000","last_change":"2024-02-18T11:47:33.516291+0000","last_active":"2024-02-18T11:47:39.556025+0000","last_peered":"2024-02-18T11:47:39.556025+0000","last_clean":"2024-02-18T11:47:39.556025+0000","last_became_active":"2024-02-18T11:47:33.516043+0000","last_became_peered":"2024-02-18T11:47:33.516043+0000","last_unstale":"2024-02-18T11:47:39.556025+0000","last_undegraded":"2024-02-18T11:47:39.556025+0000","last_fullsized":"2024-02-18T11:47:39.556025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624545+0000","last_change":"2024-02-18T11:47:34.546759+0000","last_active":"2024-02-18T11:47:39.624545+0000","last_peered":"2024-02-18T11:47:39.624545+0000","last_clean":"2024-02-18T11:47:39.624545+0000","last_became_active":"2024-02-18T11:47:34.546282+0000","last_became_peered":"2024-02-18T11:47:34.546282+0000","last_unstale":"2024-02-18T11:47:39.624545+0000","last_undegraded":"2024-02-18T11:47:39.624545+0000","last_fullsized":"2024-02-18T11:47:39.624545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808452+0000","last_change":"2024-02-18T11:47:38.571574+0000","last_active":"2024-02-18T11:47:39.808452+0000","last_peered":"2024-02-18T11:47:39.808452+0000","last_clean":"2024-02-18T11:47:39.808452+0000","last_became_active":"2024-02-18T11:47:38.571177+0000","last_became_peered":"2024-02-18T11:47:38.571177+0000","last_unstale":"2024-02-18T11:47:39.808452+0000","last_undegraded":"2024-02-18T11:47:39.808452+0000","last_fullsized":"2024-02-18T11:47:39.808452+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:29.976010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309592+0000","last_change":"2024-02-18T11:47:33.522407+0000","last_active":"2024-02-18T11:47:40.309592+0000","last_peered":"2024-02-18T11:47:40.309592+0000","last_clean":"2024-02-18T11:47:40.309592+0000","last_became_active":"2024-02-18T11:47:33.521982+0000","last_became_peered":"2024-02-18T11:47:33.521982+0000","last_unstale":"2024-02-18T11:47:40.309592+0000","last_undegraded":"2024-02-18T11:47:40.309592+0000","last_fullsized":"2024-02-18T11:47:40.309592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:54:58.132971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172176+0000","last_change":"2024-02-18T11:47:34.546074+0000","last_active":"2024-02-18T11:47:40.172176+0000","last_peered":"2024-02-18T11:47:40.172176+0000","last_clean":"2024-02-18T11:47:40.172176+0000","last_became_active":"2024-02-18T11:47:34.545447+0000","last_became_peered":"2024-02-18T11:47:34.545447+0000","last_unstale":"2024-02-18T11:47:40.172176+0000","last_undegraded":"2024-02-18T11:47:40.172176+0000","last_fullsized":"2024-02-18T11:47:40.172176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809128+0000","last_change":"2024-02-18T11:47:38.568989+0000","last_active":"2024-02-18T11:47:39.809128+0000","last_peered":"2024-02-18T11:47:39.809128+0000","last_clean":"2024-02-18T11:47:39.809128+0000","last_became_active":"2024-02-18T11:47:38.568524+0000","last_became_peered":"2024-02-18T11:47:38.568524+0000","last_unstale":"2024-02-18T11:47:39.809128+0000","last_undegraded":"2024-02-18T11:47:39.809128+0000","last_fullsized":"2024-02-18T11:47:39.809128+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309327+0000","last_change":"2024-02-18T11:47:33.527284+0000","last_active":"2024-02-18T11:47:40.309327+0000","last_peered":"2024-02-18T11:47:40.309327+0000","last_clean":"2024-02-18T11:47:40.309327+0000","last_became_active":"2024-02-18T11:47:33.526669+0000","last_became_peered":"2024-02-18T11:47:33.526669+0000","last_unstale":"2024-02-18T11:47:40.309327+0000","last_undegraded":"2024-02-18T11:47:40.309327+0000","last_fullsized":"2024-02-18T11:47:40.309327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172388+0000","last_change":"2024-02-18T11:47:34.542045+0000","last_active":"2024-02-18T11:47:40.172388+0000","last_peered":"2024-02-18T11:47:40.172388+0000","last_clean":"2024-02-18T11:47:40.172388+0000","last_became_active":"2024-02-18T11:47:34.541776+0000","last_became_peered":"2024-02-18T11:47:34.541776+0000","last_unstale":"2024-02-18T11:47:40.172388+0000","last_undegraded":"2024-02-18T11:47:40.172388+0000","last_fullsized":"2024-02-18T11:47:40.172388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808568+0000","last_change":"2024-02-18T11:47:38.570454+0000","last_active":"2024-02-18T11:47:39.808568+0000","last_peered":"2024-02-18T11:47:39.808568+0000","last_clean":"2024-02-18T11:47:39.808568+0000","last_became_active":"2024-02-18T11:47:38.569640+0000","last_became_peered":"2024-02-18T11:47:38.569640+0000","last_unstale":"2024-02-18T11:47:39.808568+0000","last_undegraded":"2024-02-18T11:47:39.808568+0000","last_fullsized":"2024-02-18T11:47:39.808568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:36.793248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556080+0000","last_change":"2024-02-18T11:47:34.534695+0000","last_active":"2024-02-18T11:47:39.556080+0000","last_peered":"2024-02-18T11:47:39.556080+0000","last_clean":"2024-02-18T11:47:39.556080+0000","last_became_active":"2024-02-18T11:47:34.534368+0000","last_became_peered":"2024-02-18T11:47:34.534368+0000","last_unstale":"2024-02-18T11:47:39.556080+0000","last_undegraded":"2024-02-18T11:47:39.556080+0000","last_fullsized":"2024-02-18T11:47:39.556080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309372+0000","last_change":"2024-02-18T11:47:33.521962+0000","last_active":"2024-02-18T11:47:40.309372+0000","last_peered":"2024-02-18T11:47:40.309372+0000","last_clean":"2024-02-18T11:47:40.309372+0000","last_became_active":"2024-02-18T11:47:33.521649+0000","last_became_peered":"2024-02-18T11:47:33.521649+0000","last_unstale":"2024-02-18T11:47:40.309372+0000","last_undegraded":"2024-02-18T11:47:40.309372+0000","last_fullsized":"2024-02-18T11:47:40.309372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:37.645450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808951+0000","last_change":"2024-02-18T11:47:38.571998+0000","last_active":"2024-02-18T11:47:39.808951+0000","last_peered":"2024-02-18T11:47:39.808951+0000","last_clean":"2024-02-18T11:47:39.808951+0000","last_became_active":"2024-02-18T11:47:38.571568+0000","last_became_peered":"2024-02-18T11:47:38.571568+0000","last_unstale":"2024-02-18T11:47:39.808951+0000","last_undegraded":"2024-02-18T11:47:39.808951+0000","last_fullsized":"2024-02-18T11:47:39.808951+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:47.822731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555491+0000","last_change":"2024-02-18T11:47:34.543371+0000","last_active":"2024-02-18T11:47:39.555491+0000","last_peered":"2024-02-18T11:47:39.555491+0000","last_clean":"2024-02-18T11:47:39.555491+0000","last_became_active":"2024-02-18T11:47:34.542194+0000","last_became_peered":"2024-02-18T11:47:34.542194+0000","last_unstale":"2024-02-18T11:47:39.555491+0000","last_undegraded":"2024-02-18T11:47:39.555491+0000","last_fullsized":"2024-02-18T11:47:39.555491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845296+0000","last_change":"2024-02-18T11:47:33.517942+0000","last_active":"2024-02-18T11:47:39.845296+0000","last_peered":"2024-02-18T11:47:39.845296+0000","last_clean":"2024-02-18T11:47:39.845296+0000","last_became_active":"2024-02-18T11:47:33.517664+0000","last_became_peered":"2024-02-18T11:47:33.517664+0000","last_unstale":"2024-02-18T11:47:39.845296+0000","last_undegraded":"2024-02-18T11:47:39.845296+0000","last_fullsized":"2024-02-18T11:47:39.845296+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808749+0000","last_change":"2024-02-18T11:47:38.573311+0000","last_active":"2024-02-18T11:47:39.808749+0000","last_peered":"2024-02-18T11:47:39.808749+0000","last_clean":"2024-02-18T11:47:39.808749+0000","last_became_active":"2024-02-18T11:47:38.572888+0000","last_became_peered":"2024-02-18T11:47:38.572888+0000","last_unstale":"2024-02-18T11:47:39.808749+0000","last_undegraded":"2024-02-18T11:47:39.808749+0000","last_fullsized":"2024-02-18T11:47:39.808749+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:51.214710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555516+0000","last_change":"2024-02-18T11:47:34.537515+0000","last_active":"2024-02-18T11:47:39.555516+0000","last_peered":"2024-02-18T11:47:39.555516+0000","last_clean":"2024-02-18T11:47:39.555516+0000","last_became_active":"2024-02-18T11:47:34.537235+0000","last_became_peered":"2024-02-18T11:47:34.537235+0000","last_unstale":"2024-02-18T11:47:39.555516+0000","last_undegraded":"2024-02-18T11:47:39.555516+0000","last_fullsized":"2024-02-18T11:47:39.555516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844871+0000","last_change":"2024-02-18T11:47:33.522206+0000","last_active":"2024-02-18T11:47:39.844871+0000","last_peered":"2024-02-18T11:47:39.844871+0000","last_clean":"2024-02-18T11:47:39.844871+0000","last_became_active":"2024-02-18T11:47:33.521878+0000","last_became_peered":"2024-02-18T11:47:33.521878+0000","last_unstale":"2024-02-18T11:47:39.844871+0000","last_undegraded":"2024-02-18T11:47:39.844871+0000","last_fullsized":"2024-02-18T11:47:39.844871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:54.867787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845354+0000","last_change":"2024-02-18T11:47:38.584826+0000","last_active":"2024-02-18T11:47:39.845354+0000","last_peered":"2024-02-18T11:47:39.845354+0000","last_clean":"2024-02-18T11:47:39.845354+0000","last_became_active":"2024-02-18T11:47:38.584161+0000","last_became_peered":"2024-02-18T11:47:38.584161+0000","last_unstale":"2024-02-18T11:47:39.845354+0000","last_undegraded":"2024-02-18T11:47:39.845354+0000","last_fullsized":"2024-02-18T11:47:39.845354+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808871+0000","last_change":"2024-02-18T11:47:34.533256+0000","last_active":"2024-02-18T11:47:39.808871+0000","last_peered":"2024-02-18T11:47:39.808871+0000","last_clean":"2024-02-18T11:47:39.808871+0000","last_became_active":"2024-02-18T11:47:34.531682+0000","last_became_peered":"2024-02-18T11:47:34.531682+0000","last_unstale":"2024-02-18T11:47:39.808871+0000","last_undegraded":"2024-02-18T11:47:39.808871+0000","last_fullsized":"2024-02-18T11:47:39.808871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:09.297073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308926+0000","last_change":"2024-02-18T11:47:33.523406+0000","last_active":"2024-02-18T11:47:40.308926+0000","last_peered":"2024-02-18T11:47:40.308926+0000","last_clean":"2024-02-18T11:47:40.308926+0000","last_became_active":"2024-02-18T11:47:33.522929+0000","last_became_peered":"2024-02-18T11:47:33.522929+0000","last_unstale":"2024-02-18T11:47:40.308926+0000","last_undegraded":"2024-02-18T11:47:40.308926+0000","last_fullsized":"2024-02-18T11:47:40.308926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809106+0000","last_change":"2024-02-18T11:47:38.572268+0000","last_active":"2024-02-18T11:47:39.809106+0000","last_peered":"2024-02-18T11:47:39.809106+0000","last_clean":"2024-02-18T11:47:39.809106+0000","last_became_active":"2024-02-18T11:47:38.571698+0000","last_became_peered":"2024-02-18T11:47:38.571698+0000","last_unstale":"2024-02-18T11:47:39.809106+0000","last_undegraded":"2024-02-18T11:47:39.809106+0000","last_fullsized":"2024-02-18T11:47:39.809106+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:59.447791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624439+0000","last_change":"2024-02-18T11:47:34.537052+0000","last_active":"2024-02-18T11:47:39.624439+0000","last_peered":"2024-02-18T11:47:39.624439+0000","last_clean":"2024-02-18T11:47:39.624439+0000","last_became_active":"2024-02-18T11:47:34.536782+0000","last_became_peered":"2024-02-18T11:47:34.536782+0000","last_unstale":"2024-02-18T11:47:39.624439+0000","last_undegraded":"2024-02-18T11:47:39.624439+0000","last_fullsized":"2024-02-18T11:47:39.624439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556919+0000","last_change":"2024-02-18T11:47:33.519918+0000","last_active":"2024-02-18T11:47:39.556919+0000","last_peered":"2024-02-18T11:47:39.556919+0000","last_clean":"2024-02-18T11:47:39.556919+0000","last_became_active":"2024-02-18T11:47:33.519651+0000","last_became_peered":"2024-02-18T11:47:33.519651+0000","last_unstale":"2024-02-18T11:47:39.556919+0000","last_undegraded":"2024-02-18T11:47:39.556919+0000","last_fullsized":"2024-02-18T11:47:39.556919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808561+0000","last_change":"2024-02-18T11:47:38.584734+0000","last_active":"2024-02-18T11:47:39.808561+0000","last_peered":"2024-02-18T11:47:39.808561+0000","last_clean":"2024-02-18T11:47:39.808561+0000","last_became_active":"2024-02-18T11:47:38.584299+0000","last_became_peered":"2024-02-18T11:47:38.584299+0000","last_unstale":"2024-02-18T11:47:39.808561+0000","last_undegraded":"2024-02-18T11:47:39.808561+0000","last_fullsized":"2024-02-18T11:47:39.808561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:59.168505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555556+0000","last_change":"2024-02-18T11:47:34.550205+0000","last_active":"2024-02-18T11:47:39.555556+0000","last_peered":"2024-02-18T11:47:39.555556+0000","last_clean":"2024-02-18T11:47:39.555556+0000","last_became_active":"2024-02-18T11:47:34.549966+0000","last_became_peered":"2024-02-18T11:47:34.549966+0000","last_unstale":"2024-02-18T11:47:39.555556+0000","last_undegraded":"2024-02-18T11:47:39.555556+0000","last_fullsized":"2024-02-18T11:47:39.555556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845365+0000","last_change":"2024-02-18T11:47:33.513489+0000","last_active":"2024-02-18T11:47:39.845365+0000","last_peered":"2024-02-18T11:47:39.845365+0000","last_clean":"2024-02-18T11:47:39.845365+0000","last_became_active":"2024-02-18T11:47:33.513181+0000","last_became_peered":"2024-02-18T11:47:33.513181+0000","last_unstale":"2024-02-18T11:47:39.845365+0000","last_undegraded":"2024-02-18T11:47:39.845365+0000","last_fullsized":"2024-02-18T11:47:39.845365+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554976+0000","last_change":"2024-02-18T11:47:38.582725+0000","last_active":"2024-02-18T11:47:39.554976+0000","last_peered":"2024-02-18T11:47:39.554976+0000","last_clean":"2024-02-18T11:47:39.554976+0000","last_became_active":"2024-02-18T11:47:38.582354+0000","last_became_peered":"2024-02-18T11:47:38.582354+0000","last_unstale":"2024-02-18T11:47:39.554976+0000","last_undegraded":"2024-02-18T11:47:39.554976+0000","last_fullsized":"2024-02-18T11:47:39.554976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845190+0000","last_change":"2024-02-18T11:47:34.543899+0000","last_active":"2024-02-18T11:47:39.845190+0000","last_peered":"2024-02-18T11:47:39.845190+0000","last_clean":"2024-02-18T11:47:39.845190+0000","last_became_active":"2024-02-18T11:47:34.543627+0000","last_became_peered":"2024-02-18T11:47:34.543627+0000","last_unstale":"2024-02-18T11:47:39.845190+0000","last_undegraded":"2024-02-18T11:47:39.845190+0000","last_fullsized":"2024-02-18T11:47:39.845190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309676+0000","last_change":"2024-02-18T11:47:33.523004+0000","last_active":"2024-02-18T11:47:40.309676+0000","last_peered":"2024-02-18T11:47:40.309676+0000","last_clean":"2024-02-18T11:47:40.309676+0000","last_became_active":"2024-02-18T11:47:33.522170+0000","last_became_peered":"2024-02-18T11:47:33.522170+0000","last_unstale":"2024-02-18T11:47:40.309676+0000","last_undegraded":"2024-02-18T11:47:40.309676+0000","last_fullsized":"2024-02-18T11:47:40.309676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:12.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624145+0000","last_change":"2024-02-18T11:47:38.582313+0000","last_active":"2024-02-18T11:47:39.624145+0000","last_peered":"2024-02-18T11:47:39.624145+0000","last_clean":"2024-02-18T11:47:39.624145+0000","last_became_active":"2024-02-18T11:47:38.581439+0000","last_became_peered":"2024-02-18T11:47:38.581439+0000","last_unstale":"2024-02-18T11:47:39.624145+0000","last_undegraded":"2024-02-18T11:47:39.624145+0000","last_fullsized":"2024-02-18T11:47:39.624145+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172507+0000","last_change":"2024-02-18T11:47:34.546405+0000","last_active":"2024-02-18T11:47:40.172507+0000","last_peered":"2024-02-18T11:47:40.172507+0000","last_clean":"2024-02-18T11:47:40.172507+0000","last_became_active":"2024-02-18T11:47:34.545110+0000","last_became_peered":"2024-02-18T11:47:34.545110+0000","last_unstale":"2024-02-18T11:47:40.172507+0000","last_undegraded":"2024-02-18T11:47:40.172507+0000","last_fullsized":"2024-02-18T11:47:40.172507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845298+0000","last_change":"2024-02-18T11:47:33.515238+0000","last_active":"2024-02-18T11:47:39.845298+0000","last_peered":"2024-02-18T11:47:39.845298+0000","last_clean":"2024-02-18T11:47:39.845298+0000","last_became_active":"2024-02-18T11:47:33.514792+0000","last_became_peered":"2024-02-18T11:47:33.514792+0000","last_unstale":"2024-02-18T11:47:39.845298+0000","last_undegraded":"2024-02-18T11:47:39.845298+0000","last_fullsized":"2024-02-18T11:47:39.845298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845086+0000","last_change":"2024-02-18T11:47:38.569586+0000","last_active":"2024-02-18T11:47:39.845086+0000","last_peered":"2024-02-18T11:47:39.845086+0000","last_clean":"2024-02-18T11:47:39.845086+0000","last_became_active":"2024-02-18T11:47:38.568487+0000","last_became_peered":"2024-02-18T11:47:38.568487+0000","last_unstale":"2024-02-18T11:47:39.845086+0000","last_undegraded":"2024-02-18T11:47:39.845086+0000","last_fullsized":"2024-02-18T11:47:39.845086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809233+0000","last_change":"2024-02-18T11:47:34.533351+0000","last_active":"2024-02-18T11:47:39.809233+0000","last_peered":"2024-02-18T11:47:39.809233+0000","last_clean":"2024-02-18T11:47:39.809233+0000","last_became_active":"2024-02-18T11:47:34.532667+0000","last_became_peered":"2024-02-18T11:47:34.532667+0000","last_unstale":"2024-02-18T11:47:39.809233+0000","last_undegraded":"2024-02-18T11:47:39.809233+0000","last_fullsized":"2024-02-18T11:47:39.809233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:26.230805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172223+0000","last_change":"2024-02-18T11:47:33.526859+0000","last_active":"2024-02-18T11:47:40.172223+0000","last_peered":"2024-02-18T11:47:40.172223+0000","last_clean":"2024-02-18T11:47:40.172223+0000","last_became_active":"2024-02-18T11:47:33.525839+0000","last_became_peered":"2024-02-18T11:47:33.525839+0000","last_unstale":"2024-02-18T11:47:40.172223+0000","last_undegraded":"2024-02-18T11:47:40.172223+0000","last_fullsized":"2024-02-18T11:47:40.172223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845416+0000","last_change":"2024-02-18T11:47:38.562544+0000","last_active":"2024-02-18T11:47:39.845416+0000","last_peered":"2024-02-18T11:47:39.845416+0000","last_clean":"2024-02-18T11:47:39.845416+0000","last_became_active":"2024-02-18T11:47:38.562171+0000","last_became_peered":"2024-02-18T11:47:38.562171+0000","last_unstale":"2024-02-18T11:47:39.845416+0000","last_undegraded":"2024-02-18T11:47:39.845416+0000","last_fullsized":"2024-02-18T11:47:39.845416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844931+0000","last_change":"2024-02-18T11:47:34.544098+0000","last_active":"2024-02-18T11:47:39.844931+0000","last_peered":"2024-02-18T11:47:39.844931+0000","last_clean":"2024-02-18T11:47:39.844931+0000","last_became_active":"2024-02-18T11:47:34.543279+0000","last_became_peered":"2024-02-18T11:47:34.543279+0000","last_unstale":"2024-02-18T11:47:39.844931+0000","last_undegraded":"2024-02-18T11:47:39.844931+0000","last_fullsized":"2024-02-18T11:47:39.844931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554630+0000","last_change":"2024-02-18T11:47:33.518923+0000","last_active":"2024-02-18T11:47:39.554630+0000","last_peered":"2024-02-18T11:47:39.554630+0000","last_clean":"2024-02-18T11:47:39.554630+0000","last_became_active":"2024-02-18T11:47:33.518591+0000","last_became_peered":"2024-02-18T11:47:33.518591+0000","last_unstale":"2024-02-18T11:47:39.554630+0000","last_undegraded":"2024-02-18T11:47:39.554630+0000","last_fullsized":"2024-02-18T11:47:39.554630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845451+0000","last_change":"2024-02-18T11:47:38.571577+0000","last_active":"2024-02-18T11:47:39.845451+0000","last_peered":"2024-02-18T11:47:39.845451+0000","last_clean":"2024-02-18T11:47:39.845451+0000","last_became_active":"2024-02-18T11:47:38.571165+0000","last_became_peered":"2024-02-18T11:47:38.571165+0000","last_unstale":"2024-02-18T11:47:39.845451+0000","last_undegraded":"2024-02-18T11:47:39.845451+0000","last_fullsized":"2024-02-18T11:47:39.845451+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.623861+0000","last_change":"2024-02-18T11:47:34.533365+0000","last_active":"2024-02-18T11:47:39.623861+0000","last_peered":"2024-02-18T11:47:39.623861+0000","last_clean":"2024-02-18T11:47:39.623861+0000","last_became_active":"2024-02-18T11:47:34.533103+0000","last_became_peered":"2024-02-18T11:47:34.533103+0000","last_unstale":"2024-02-18T11:47:39.623861+0000","last_undegraded":"2024-02-18T11:47:39.623861+0000","last_fullsized":"2024-02-18T11:47:39.623861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:27:13.022192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.308945+0000","last_change":"2024-02-18T11:47:33.523599+0000","last_active":"2024-02-18T11:47:40.308945+0000","last_peered":"2024-02-18T11:47:40.308945+0000","last_clean":"2024-02-18T11:47:40.308945+0000","last_became_active":"2024-02-18T11:47:33.523327+0000","last_became_peered":"2024-02-18T11:47:33.523327+0000","last_unstale":"2024-02-18T11:47:40.308945+0000","last_undegraded":"2024-02-18T11:47:40.308945+0000","last_fullsized":"2024-02-18T11:47:40.308945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624414+0000","last_change":"2024-02-18T11:47:38.581803+0000","last_active":"2024-02-18T11:47:39.624414+0000","last_peered":"2024-02-18T11:47:39.624414+0000","last_clean":"2024-02-18T11:47:39.624414+0000","last_became_active":"2024-02-18T11:47:38.581401+0000","last_became_peered":"2024-02-18T11:47:38.581401+0000","last_unstale":"2024-02-18T11:47:39.624414+0000","last_undegraded":"2024-02-18T11:47:39.624414+0000","last_fullsized":"2024-02-18T11:47:39.624414+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:01:15.281283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556062+0000","last_change":"2024-02-18T11:47:34.544673+0000","last_active":"2024-02-18T11:47:39.556062+0000","last_peered":"2024-02-18T11:47:39.556062+0000","last_clean":"2024-02-18T11:47:39.556062+0000","last_became_active":"2024-02-18T11:47:34.544427+0000","last_became_peered":"2024-02-18T11:47:34.544427+0000","last_unstale":"2024-02-18T11:47:39.556062+0000","last_undegraded":"2024-02-18T11:47:39.556062+0000","last_fullsized":"2024-02-18T11:47:39.556062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172045+0000","last_change":"2024-02-18T11:47:33.522514+0000","last_active":"2024-02-18T11:47:40.172045+0000","last_peered":"2024-02-18T11:47:40.172045+0000","last_clean":"2024-02-18T11:47:40.172045+0000","last_became_active":"2024-02-18T11:47:33.522205+0000","last_became_peered":"2024-02-18T11:47:33.522205+0000","last_unstale":"2024-02-18T11:47:40.172045+0000","last_undegraded":"2024-02-18T11:47:40.172045+0000","last_fullsized":"2024-02-18T11:47:40.172045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309116+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:47:40.309116+0000","last_peered":"2024-02-18T11:47:40.309116+0000","last_clean":"2024-02-18T11:47:40.309116+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:47:40.309116+0000","last_undegraded":"2024-02-18T11:47:40.309116+0000","last_fullsized":"2024-02-18T11:47:40.309116+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556628+0000","last_change":"2024-02-18T11:47:38.584484+0000","last_active":"2024-02-18T11:47:39.556628+0000","last_peered":"2024-02-18T11:47:39.556628+0000","last_clean":"2024-02-18T11:47:39.556628+0000","last_became_active":"2024-02-18T11:47:38.584089+0000","last_became_peered":"2024-02-18T11:47:38.584089+0000","last_unstale":"2024-02-18T11:47:39.556628+0000","last_undegraded":"2024-02-18T11:47:39.556628+0000","last_fullsized":"2024-02-18T11:47:39.556628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:10.343777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309440+0000","last_change":"2024-02-18T11:47:34.548586+0000","last_active":"2024-02-18T11:47:40.309440+0000","last_peered":"2024-02-18T11:47:40.309440+0000","last_clean":"2024-02-18T11:47:40.309440+0000","last_became_active":"2024-02-18T11:47:34.548315+0000","last_became_peered":"2024-02-18T11:47:34.548315+0000","last_unstale":"2024-02-18T11:47:40.309440+0000","last_undegraded":"2024-02-18T11:47:40.309440+0000","last_fullsized":"2024-02-18T11:47:40.309440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:36.903173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554890+0000","last_change":"2024-02-18T11:47:33.526959+0000","last_active":"2024-02-18T11:47:39.554890+0000","last_peered":"2024-02-18T11:47:39.554890+0000","last_clean":"2024-02-18T11:47:39.554890+0000","last_became_active":"2024-02-18T11:47:33.526632+0000","last_became_peered":"2024-02-18T11:47:33.526632+0000","last_unstale":"2024-02-18T11:47:39.554890+0000","last_undegraded":"2024-02-18T11:47:39.554890+0000","last_fullsized":"2024-02-18T11:47:39.554890+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:40:26.815930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556483+0000","last_change":"2024-02-18T11:47:38.581910+0000","last_active":"2024-02-18T11:47:39.556483+0000","last_peered":"2024-02-18T11:47:39.556483+0000","last_clean":"2024-02-18T11:47:39.556483+0000","last_became_active":"2024-02-18T11:47:38.581463+0000","last_became_peered":"2024-02-18T11:47:38.581463+0000","last_unstale":"2024-02-18T11:47:39.556483+0000","last_undegraded":"2024-02-18T11:47:39.556483+0000","last_fullsized":"2024-02-18T11:47:39.556483+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172330+0000","last_change":"2024-02-18T11:47:34.545863+0000","last_active":"2024-02-18T11:47:40.172330+0000","last_peered":"2024-02-18T11:47:40.172330+0000","last_clean":"2024-02-18T11:47:40.172330+0000","last_became_active":"2024-02-18T11:47:34.545142+0000","last_became_peered":"2024-02-18T11:47:34.545142+0000","last_unstale":"2024-02-18T11:47:40.172330+0000","last_undegraded":"2024-02-18T11:47:40.172330+0000","last_fullsized":"2024-02-18T11:47:40.172330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845452+0000","last_change":"2024-02-18T11:47:33.519784+0000","last_active":"2024-02-18T11:47:39.845452+0000","last_peered":"2024-02-18T11:47:39.845452+0000","last_clean":"2024-02-18T11:47:39.845452+0000","last_became_active":"2024-02-18T11:47:33.519495+0000","last_became_peered":"2024-02-18T11:47:33.519495+0000","last_unstale":"2024-02-18T11:47:39.845452+0000","last_undegraded":"2024-02-18T11:47:39.845452+0000","last_fullsized":"2024-02-18T11:47:39.845452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624233+0000","last_change":"2024-02-18T11:47:38.575385+0000","last_active":"2024-02-18T11:47:39.624233+0000","last_peered":"2024-02-18T11:47:39.624233+0000","last_clean":"2024-02-18T11:47:39.624233+0000","last_became_active":"2024-02-18T11:47:38.574974+0000","last_became_peered":"2024-02-18T11:47:38.574974+0000","last_unstale":"2024-02-18T11:47:39.624233+0000","last_undegraded":"2024-02-18T11:47:39.624233+0000","last_fullsized":"2024-02-18T11:47:39.624233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:25.907852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309750+0000","last_change":"2024-02-18T11:47:34.550657+0000","last_active":"2024-02-18T11:47:40.309750+0000","last_peered":"2024-02-18T11:47:40.309750+0000","last_clean":"2024-02-18T11:47:40.309750+0000","last_became_active":"2024-02-18T11:47:34.550286+0000","last_became_peered":"2024-02-18T11:47:34.550286+0000","last_unstale":"2024-02-18T11:47:40.309750+0000","last_undegraded":"2024-02-18T11:47:40.309750+0000","last_fullsized":"2024-02-18T11:47:40.309750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:15.797810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555138+0000","last_change":"2024-02-18T11:47:33.518941+0000","last_active":"2024-02-18T11:47:39.555138+0000","last_peered":"2024-02-18T11:47:39.555138+0000","last_clean":"2024-02-18T11:47:39.555138+0000","last_became_active":"2024-02-18T11:47:33.518644+0000","last_became_peered":"2024-02-18T11:47:33.518644+0000","last_unstale":"2024-02-18T11:47:39.555138+0000","last_undegraded":"2024-02-18T11:47:39.555138+0000","last_fullsized":"2024-02-18T11:47:39.555138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:17:08.290395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171756+0000","last_change":"2024-02-18T11:47:38.582832+0000","last_active":"2024-02-18T11:47:40.171756+0000","last_peered":"2024-02-18T11:47:40.171756+0000","last_clean":"2024-02-18T11:47:40.171756+0000","last_became_active":"2024-02-18T11:47:38.582436+0000","last_became_peered":"2024-02-18T11:47:38.582436+0000","last_unstale":"2024-02-18T11:47:40.171756+0000","last_undegraded":"2024-02-18T11:47:40.171756+0000","last_fullsized":"2024-02-18T11:47:40.171756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309643+0000","last_change":"2024-02-18T11:47:34.544607+0000","last_active":"2024-02-18T11:47:40.309643+0000","last_peered":"2024-02-18T11:47:40.309643+0000","last_clean":"2024-02-18T11:47:40.309643+0000","last_became_active":"2024-02-18T11:47:34.544257+0000","last_became_peered":"2024-02-18T11:47:34.544257+0000","last_unstale":"2024-02-18T11:47:40.309643+0000","last_undegraded":"2024-02-18T11:47:40.309643+0000","last_fullsized":"2024-02-18T11:47:40.309643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:25:06.300794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809144+0000","last_change":"2024-02-18T11:47:33.516233+0000","last_active":"2024-02-18T11:47:39.809144+0000","last_peered":"2024-02-18T11:47:39.809144+0000","last_clean":"2024-02-18T11:47:39.809144+0000","last_became_active":"2024-02-18T11:47:33.515830+0000","last_became_peered":"2024-02-18T11:47:33.515830+0000","last_unstale":"2024-02-18T11:47:39.809144+0000","last_undegraded":"2024-02-18T11:47:39.809144+0000","last_fullsized":"2024-02-18T11:47:39.809144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:43:01.501097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808439+0000","last_change":"2024-02-18T11:47:38.573381+0000","last_active":"2024-02-18T11:47:39.808439+0000","last_peered":"2024-02-18T11:47:39.808439+0000","last_clean":"2024-02-18T11:47:39.808439+0000","last_became_active":"2024-02-18T11:47:38.572824+0000","last_became_peered":"2024-02-18T11:47:38.572824+0000","last_unstale":"2024-02-18T11:47:39.808439+0000","last_undegraded":"2024-02-18T11:47:39.808439+0000","last_fullsized":"2024-02-18T11:47:39.808439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:11.073343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309074+0000","last_change":"2024-02-18T11:47:34.543813+0000","last_active":"2024-02-18T11:47:40.309074+0000","last_peered":"2024-02-18T11:47:40.309074+0000","last_clean":"2024-02-18T11:47:40.309074+0000","last_became_active":"2024-02-18T11:47:34.543520+0000","last_became_peered":"2024-02-18T11:47:34.543520+0000","last_unstale":"2024-02-18T11:47:40.309074+0000","last_undegraded":"2024-02-18T11:47:40.309074+0000","last_fullsized":"2024-02-18T11:47:40.309074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554409+0000","last_change":"2024-02-18T11:47:33.519311+0000","last_active":"2024-02-18T11:47:39.554409+0000","last_peered":"2024-02-18T11:47:39.554409+0000","last_clean":"2024-02-18T11:47:39.554409+0000","last_became_active":"2024-02-18T11:47:33.518936+0000","last_became_peered":"2024-02-18T11:47:33.518936+0000","last_unstale":"2024-02-18T11:47:39.554409+0000","last_undegraded":"2024-02-18T11:47:39.554409+0000","last_fullsized":"2024-02-18T11:47:39.554409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624339+0000","last_change":"2024-02-18T11:47:38.568770+0000","last_active":"2024-02-18T11:47:39.624339+0000","last_peered":"2024-02-18T11:47:39.624339+0000","last_clean":"2024-02-18T11:47:39.624339+0000","last_became_active":"2024-02-18T11:47:38.568394+0000","last_became_peered":"2024-02-18T11:47:38.568394+0000","last_unstale":"2024-02-18T11:47:39.624339+0000","last_undegraded":"2024-02-18T11:47:39.624339+0000","last_fullsized":"2024-02-18T11:47:39.624339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:04.112912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556127+0000","last_change":"2024-02-18T11:47:34.530166+0000","last_active":"2024-02-18T11:47:39.556127+0000","last_peered":"2024-02-18T11:47:39.556127+0000","last_clean":"2024-02-18T11:47:39.556127+0000","last_became_active":"2024-02-18T11:47:34.529875+0000","last_became_peered":"2024-02-18T11:47:34.529875+0000","last_unstale":"2024-02-18T11:47:39.556127+0000","last_undegraded":"2024-02-18T11:47:39.556127+0000","last_fullsized":"2024-02-18T11:47:39.556127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556190+0000","last_change":"2024-02-18T11:47:33.519858+0000","last_active":"2024-02-18T11:47:39.556190+0000","last_peered":"2024-02-18T11:47:39.556190+0000","last_clean":"2024-02-18T11:47:39.556190+0000","last_became_active":"2024-02-18T11:47:33.519239+0000","last_became_peered":"2024-02-18T11:47:33.519239+0000","last_unstale":"2024-02-18T11:47:39.556190+0000","last_undegraded":"2024-02-18T11:47:39.556190+0000","last_fullsized":"2024-02-18T11:47:39.556190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555696+0000","last_change":"2024-02-18T11:47:38.583646+0000","last_active":"2024-02-18T11:47:39.555696+0000","last_peered":"2024-02-18T11:47:39.555696+0000","last_clean":"2024-02-18T11:47:39.555696+0000","last_became_active":"2024-02-18T11:47:38.583283+0000","last_became_peered":"2024-02-18T11:47:38.583283+0000","last_unstale":"2024-02-18T11:47:39.555696+0000","last_undegraded":"2024-02-18T11:47:39.555696+0000","last_fullsized":"2024-02-18T11:47:39.555696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:13:48.369727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309237+0000","last_change":"2024-02-18T11:47:34.549395+0000","last_active":"2024-02-18T11:47:40.309237+0000","last_peered":"2024-02-18T11:47:40.309237+0000","last_clean":"2024-02-18T11:47:40.309237+0000","last_became_active":"2024-02-18T11:47:34.549157+0000","last_became_peered":"2024-02-18T11:47:34.549157+0000","last_unstale":"2024-02-18T11:47:40.309237+0000","last_undegraded":"2024-02-18T11:47:40.309237+0000","last_fullsized":"2024-02-18T11:47:40.309237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172210+0000","last_change":"2024-02-18T11:47:33.526266+0000","last_active":"2024-02-18T11:47:40.172210+0000","last_peered":"2024-02-18T11:47:40.172210+0000","last_clean":"2024-02-18T11:47:40.172210+0000","last_became_active":"2024-02-18T11:47:33.525885+0000","last_became_peered":"2024-02-18T11:47:33.525885+0000","last_unstale":"2024-02-18T11:47:40.172210+0000","last_undegraded":"2024-02-18T11:47:40.172210+0000","last_fullsized":"2024-02-18T11:47:40.172210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555736+0000","last_change":"2024-02-18T11:47:38.581784+0000","last_active":"2024-02-18T11:47:39.555736+0000","last_peered":"2024-02-18T11:47:39.555736+0000","last_clean":"2024-02-18T11:47:39.555736+0000","last_became_active":"2024-02-18T11:47:38.581391+0000","last_became_peered":"2024-02-18T11:47:38.581391+0000","last_unstale":"2024-02-18T11:47:39.555736+0000","last_undegraded":"2024-02-18T11:47:39.555736+0000","last_fullsized":"2024-02-18T11:47:39.555736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624104+0000","last_change":"2024-02-18T11:47:34.537306+0000","last_active":"2024-02-18T11:47:39.624104+0000","last_peered":"2024-02-18T11:47:39.624104+0000","last_clean":"2024-02-18T11:47:39.624104+0000","last_became_active":"2024-02-18T11:47:34.537048+0000","last_became_peered":"2024-02-18T11:47:34.537048+0000","last_unstale":"2024-02-18T11:47:39.624104+0000","last_undegraded":"2024-02-18T11:47:39.624104+0000","last_fullsized":"2024-02-18T11:47:39.624104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:19:55.096952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624290+0000","last_change":"2024-02-18T11:47:33.518377+0000","last_active":"2024-02-18T11:47:39.624290+0000","last_peered":"2024-02-18T11:47:39.624290+0000","last_clean":"2024-02-18T11:47:39.624290+0000","last_became_active":"2024-02-18T11:47:33.518105+0000","last_became_peered":"2024-02-18T11:47:33.518105+0000","last_unstale":"2024-02-18T11:47:39.624290+0000","last_undegraded":"2024-02-18T11:47:39.624290+0000","last_fullsized":"2024-02-18T11:47:39.624290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:12:42.834778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172424+0000","last_change":"2024-02-18T11:47:38.585105+0000","last_active":"2024-02-18T11:47:40.172424+0000","last_peered":"2024-02-18T11:47:40.172424+0000","last_clean":"2024-02-18T11:47:40.172424+0000","last_became_active":"2024-02-18T11:47:38.584660+0000","last_became_peered":"2024-02-18T11:47:38.584660+0000","last_unstale":"2024-02-18T11:47:40.172424+0000","last_undegraded":"2024-02-18T11:47:40.172424+0000","last_fullsized":"2024-02-18T11:47:40.172424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:51.793495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309312+0000","last_change":"2024-02-18T11:47:34.546219+0000","last_active":"2024-02-18T11:47:40.309312+0000","last_peered":"2024-02-18T11:47:40.309312+0000","last_clean":"2024-02-18T11:47:40.309312+0000","last_became_active":"2024-02-18T11:47:34.545961+0000","last_became_peered":"2024-02-18T11:47:34.545961+0000","last_unstale":"2024-02-18T11:47:40.309312+0000","last_undegraded":"2024-02-18T11:47:40.309312+0000","last_fullsized":"2024-02-18T11:47:40.309312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:43.783562+0000","last_change":"2024-02-18T11:47:33.511834+0000","last_active":"2024-02-18T11:47:43.783562+0000","last_peered":"2024-02-18T11:47:43.783562+0000","last_clean":"2024-02-18T11:47:43.783562+0000","last_became_active":"2024-02-18T11:47:33.511537+0000","last_became_peered":"2024-02-18T11:47:33.511537+0000","last_unstale":"2024-02-18T11:47:43.783562+0000","last_undegraded":"2024-02-18T11:47:43.783562+0000","last_fullsized":"2024-02-18T11:47:43.783562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845016+0000","last_change":"2024-02-18T11:47:38.570461+0000","last_active":"2024-02-18T11:47:39.845016+0000","last_peered":"2024-02-18T11:47:39.845016+0000","last_clean":"2024-02-18T11:47:39.845016+0000","last_became_active":"2024-02-18T11:47:38.568615+0000","last_became_peered":"2024-02-18T11:47:38.568615+0000","last_unstale":"2024-02-18T11:47:39.845016+0000","last_undegraded":"2024-02-18T11:47:39.845016+0000","last_fullsized":"2024-02-18T11:47:39.845016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:52:40.946156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808848+0000","last_change":"2024-02-18T11:47:34.541278+0000","last_active":"2024-02-18T11:47:39.808848+0000","last_peered":"2024-02-18T11:47:39.808848+0000","last_clean":"2024-02-18T11:47:39.808848+0000","last_became_active":"2024-02-18T11:47:34.540891+0000","last_became_peered":"2024-02-18T11:47:34.540891+0000","last_unstale":"2024-02-18T11:47:39.808848+0000","last_undegraded":"2024-02-18T11:47:39.808848+0000","last_fullsized":"2024-02-18T11:47:39.808848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845415+0000","last_change":"2024-02-18T11:47:33.526021+0000","last_active":"2024-02-18T11:47:39.845415+0000","last_peered":"2024-02-18T11:47:39.845415+0000","last_clean":"2024-02-18T11:47:39.845415+0000","last_became_active":"2024-02-18T11:47:33.525810+0000","last_became_peered":"2024-02-18T11:47:33.525810+0000","last_unstale":"2024-02-18T11:47:39.845415+0000","last_undegraded":"2024-02-18T11:47:39.845415+0000","last_fullsized":"2024-02-18T11:47:39.845415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:15:21.866225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.809063+0000","last_change":"2024-02-18T11:47:38.568064+0000","last_active":"2024-02-18T11:47:39.809063+0000","last_peered":"2024-02-18T11:47:39.809063+0000","last_clean":"2024-02-18T11:47:39.809063+0000","last_became_active":"2024-02-18T11:47:38.567648+0000","last_became_peered":"2024-02-18T11:47:38.567648+0000","last_unstale":"2024-02-18T11:47:39.809063+0000","last_undegraded":"2024-02-18T11:47:39.809063+0000","last_fullsized":"2024-02-18T11:47:39.809063+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:41.335475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624622+0000","last_change":"2024-02-18T11:47:34.546545+0000","last_active":"2024-02-18T11:47:39.624622+0000","last_peered":"2024-02-18T11:47:39.624622+0000","last_clean":"2024-02-18T11:47:39.624622+0000","last_became_active":"2024-02-18T11:47:34.545918+0000","last_became_peered":"2024-02-18T11:47:34.545918+0000","last_unstale":"2024-02-18T11:47:39.624622+0000","last_undegraded":"2024-02-18T11:47:39.624622+0000","last_fullsized":"2024-02-18T11:47:39.624622+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172369+0000","last_change":"2024-02-18T11:47:33.526119+0000","last_active":"2024-02-18T11:47:40.172369+0000","last_peered":"2024-02-18T11:47:40.172369+0000","last_clean":"2024-02-18T11:47:40.172369+0000","last_became_active":"2024-02-18T11:47:33.525722+0000","last_became_peered":"2024-02-18T11:47:33.525722+0000","last_unstale":"2024-02-18T11:47:40.172369+0000","last_undegraded":"2024-02-18T11:47:40.172369+0000","last_fullsized":"2024-02-18T11:47:40.172369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555321+0000","last_change":"2024-02-18T11:47:38.569853+0000","last_active":"2024-02-18T11:47:39.555321+0000","last_peered":"2024-02-18T11:47:39.555321+0000","last_clean":"2024-02-18T11:47:39.555321+0000","last_became_active":"2024-02-18T11:47:38.569315+0000","last_became_peered":"2024-02-18T11:47:38.569315+0000","last_unstale":"2024-02-18T11:47:39.555321+0000","last_undegraded":"2024-02-18T11:47:39.555321+0000","last_fullsized":"2024-02-18T11:47:39.555321+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:22:24.799952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172285+0000","last_change":"2024-02-18T11:47:34.538399+0000","last_active":"2024-02-18T11:47:40.172285+0000","last_peered":"2024-02-18T11:47:40.172285+0000","last_clean":"2024-02-18T11:47:40.172285+0000","last_became_active":"2024-02-18T11:47:34.538133+0000","last_became_peered":"2024-02-18T11:47:34.538133+0000","last_unstale":"2024-02-18T11:47:40.172285+0000","last_undegraded":"2024-02-18T11:47:40.172285+0000","last_fullsized":"2024-02-18T11:47:40.172285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556798+0000","last_change":"2024-02-18T11:47:33.526252+0000","last_active":"2024-02-18T11:47:39.556798+0000","last_peered":"2024-02-18T11:47:39.556798+0000","last_clean":"2024-02-18T11:47:39.556798+0000","last_became_active":"2024-02-18T11:47:33.525952+0000","last_became_peered":"2024-02-18T11:47:33.525952+0000","last_unstale":"2024-02-18T11:47:39.556798+0000","last_undegraded":"2024-02-18T11:47:39.556798+0000","last_fullsized":"2024-02-18T11:47:39.556798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:57.798403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555129+0000","last_change":"2024-02-18T11:47:38.583952+0000","last_active":"2024-02-18T11:47:39.555129+0000","last_peered":"2024-02-18T11:47:39.555129+0000","last_clean":"2024-02-18T11:47:39.555129+0000","last_became_active":"2024-02-18T11:47:38.583579+0000","last_became_peered":"2024-02-18T11:47:38.583579+0000","last_unstale":"2024-02-18T11:47:39.555129+0000","last_undegraded":"2024-02-18T11:47:39.555129+0000","last_fullsized":"2024-02-18T11:47:39.555129+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:33:29.418521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624283+0000","last_change":"2024-02-18T11:47:38.583643+0000","last_active":"2024-02-18T11:47:39.624283+0000","last_peered":"2024-02-18T11:47:39.624283+0000","last_clean":"2024-02-18T11:47:39.624283+0000","last_became_active":"2024-02-18T11:47:38.583245+0000","last_became_peered":"2024-02-18T11:47:38.583245+0000","last_unstale":"2024-02-18T11:47:39.624283+0000","last_undegraded":"2024-02-18T11:47:39.624283+0000","last_fullsized":"2024-02-18T11:47:39.624283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:38:19.840793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844764+0000","last_change":"2024-02-18T11:47:34.540661+0000","last_active":"2024-02-18T11:47:39.844764+0000","last_peered":"2024-02-18T11:47:39.844764+0000","last_clean":"2024-02-18T11:47:39.844764+0000","last_became_active":"2024-02-18T11:47:34.540342+0000","last_became_peered":"2024-02-18T11:47:34.540342+0000","last_unstale":"2024-02-18T11:47:39.844764+0000","last_undegraded":"2024-02-18T11:47:39.844764+0000","last_fullsized":"2024-02-18T11:47:39.844764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171940+0000","last_change":"2024-02-18T11:47:33.526132+0000","last_active":"2024-02-18T11:47:40.171940+0000","last_peered":"2024-02-18T11:47:40.171940+0000","last_clean":"2024-02-18T11:47:40.171940+0000","last_became_active":"2024-02-18T11:47:33.525798+0000","last_became_peered":"2024-02-18T11:47:33.525798+0000","last_unstale":"2024-02-18T11:47:40.171940+0000","last_undegraded":"2024-02-18T11:47:40.171940+0000","last_fullsized":"2024-02-18T11:47:40.171940+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:42.011483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556839+0000","last_change":"2024-02-18T11:47:38.572273+0000","last_active":"2024-02-18T11:47:39.556839+0000","last_peered":"2024-02-18T11:47:39.556839+0000","last_clean":"2024-02-18T11:47:39.556839+0000","last_became_active":"2024-02-18T11:47:38.571852+0000","last_became_peered":"2024-02-18T11:47:38.571852+0000","last_unstale":"2024-02-18T11:47:39.556839+0000","last_undegraded":"2024-02-18T11:47:39.556839+0000","last_fullsized":"2024-02-18T11:47:39.556839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:09.789796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555386+0000","last_change":"2024-02-18T11:47:34.546984+0000","last_active":"2024-02-18T11:47:39.555386+0000","last_peered":"2024-02-18T11:47:39.555386+0000","last_clean":"2024-02-18T11:47:39.555386+0000","last_became_active":"2024-02-18T11:47:34.546738+0000","last_became_peered":"2024-02-18T11:47:34.546738+0000","last_unstale":"2024-02-18T11:47:39.555386+0000","last_undegraded":"2024-02-18T11:47:39.555386+0000","last_fullsized":"2024-02-18T11:47:39.555386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:07:56.815908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555662+0000","last_change":"2024-02-18T11:47:33.514025+0000","last_active":"2024-02-18T11:47:39.555662+0000","last_peered":"2024-02-18T11:47:39.555662+0000","last_clean":"2024-02-18T11:47:39.555662+0000","last_became_active":"2024-02-18T11:47:33.513762+0000","last_became_peered":"2024-02-18T11:47:33.513762+0000","last_unstale":"2024-02-18T11:47:39.555662+0000","last_undegraded":"2024-02-18T11:47:39.555662+0000","last_fullsized":"2024-02-18T11:47:39.555662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:28:17.454510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556529+0000","last_change":"2024-02-18T11:47:33.521856+0000","last_active":"2024-02-18T11:47:39.556529+0000","last_peered":"2024-02-18T11:47:39.556529+0000","last_clean":"2024-02-18T11:47:39.556529+0000","last_became_active":"2024-02-18T11:47:33.521563+0000","last_became_peered":"2024-02-18T11:47:33.521563+0000","last_unstale":"2024-02-18T11:47:39.556529+0000","last_undegraded":"2024-02-18T11:47:39.556529+0000","last_fullsized":"2024-02-18T11:47:39.556529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:53:25.224127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624276+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.624276+0000","last_peered":"2024-02-18T11:47:39.624276+0000","last_clean":"2024-02-18T11:47:39.624276+0000","last_became_active":"2024-02-18T11:47:34.536344+0000","last_became_peered":"2024-02-18T11:47:34.536344+0000","last_unstale":"2024-02-18T11:47:39.624276+0000","last_undegraded":"2024-02-18T11:47:39.624276+0000","last_fullsized":"2024-02-18T11:47:39.624276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:42:19.273337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554790+0000","last_change":"2024-02-18T11:47:38.572780+0000","last_active":"2024-02-18T11:47:39.554790+0000","last_peered":"2024-02-18T11:47:39.554790+0000","last_clean":"2024-02-18T11:47:39.554790+0000","last_became_active":"2024-02-18T11:47:38.572401+0000","last_became_peered":"2024-02-18T11:47:38.572401+0000","last_unstale":"2024-02-18T11:47:39.554790+0000","last_undegraded":"2024-02-18T11:47:39.554790+0000","last_fullsized":"2024-02-18T11:47:39.554790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:06.469768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555607+0000","last_change":"2024-02-18T11:47:33.521346+0000","last_active":"2024-02-18T11:47:39.555607+0000","last_peered":"2024-02-18T11:47:39.555607+0000","last_clean":"2024-02-18T11:47:39.555607+0000","last_became_active":"2024-02-18T11:47:33.521077+0000","last_became_peered":"2024-02-18T11:47:33.521077+0000","last_unstale":"2024-02-18T11:47:39.555607+0000","last_undegraded":"2024-02-18T11:47:39.555607+0000","last_fullsized":"2024-02-18T11:47:39.555607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:32.492794+0000","last_clean_scrub_stamp":"2024-02-18T11:47:32.492794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:00:41.989789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808655+0000","last_change":"2024-02-18T11:47:34.535139+0000","last_active":"2024-02-18T11:47:39.808655+0000","last_peered":"2024-02-18T11:47:39.808655+0000","last_clean":"2024-02-18T11:47:39.808655+0000","last_became_active":"2024-02-18T11:47:34.534871+0000","last_became_peered":"2024-02-18T11:47:34.534871+0000","last_unstale":"2024-02-18T11:47:39.808655+0000","last_undegraded":"2024-02-18T11:47:39.808655+0000","last_fullsized":"2024-02-18T11:47:39.808655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:04:58.805082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556009+0000","last_change":"2024-02-18T11:47:38.566494+0000","last_active":"2024-02-18T11:47:39.556009+0000","last_peered":"2024-02-18T11:47:39.556009+0000","last_clean":"2024-02-18T11:47:39.556009+0000","last_became_active":"2024-02-18T11:47:38.565672+0000","last_became_peered":"2024-02-18T11:47:38.565672+0000","last_unstale":"2024-02-18T11:47:39.556009+0000","last_undegraded":"2024-02-18T11:47:39.556009+0000","last_fullsized":"2024-02-18T11:47:39.556009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:37.542455+0000","last_clean_scrub_stamp":"2024-02-18T11:47:37.542455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:38:29.952609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556029+0000","last_change":"2024-02-18T11:47:34.531011+0000","last_active":"2024-02-18T11:47:39.556029+0000","last_peered":"2024-02-18T11:47:39.556029+0000","last_clean":"2024-02-18T11:47:39.556029+0000","last_became_active":"2024-02-18T11:47:34.530758+0000","last_became_peered":"2024-02-18T11:47:34.530758+0000","last_unstale":"2024-02-18T11:47:39.556029+0000","last_undegraded":"2024-02-18T11:47:39.556029+0000","last_fullsized":"2024-02-18T11:47:39.556029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.554518+0000","last_change":"2024-02-18T11:47:34.543755+0000","last_active":"2024-02-18T11:47:39.554518+0000","last_peered":"2024-02-18T11:47:39.554518+0000","last_clean":"2024-02-18T11:47:39.554518+0000","last_became_active":"2024-02-18T11:47:34.543500+0000","last_became_peered":"2024-02-18T11:47:34.543500+0000","last_unstale":"2024-02-18T11:47:39.554518+0000","last_undegraded":"2024-02-18T11:47:39.554518+0000","last_fullsized":"2024-02-18T11:47:39.554518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:43:55.874466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172079+0000","last_change":"2024-02-18T11:47:34.545316+0000","last_active":"2024-02-18T11:47:40.172079+0000","last_peered":"2024-02-18T11:47:40.172079+0000","last_clean":"2024-02-18T11:47:40.172079+0000","last_became_active":"2024-02-18T11:47:34.545018+0000","last_became_peered":"2024-02-18T11:47:34.545018+0000","last_unstale":"2024-02-18T11:47:40.172079+0000","last_undegraded":"2024-02-18T11:47:40.172079+0000","last_fullsized":"2024-02-18T11:47:40.172079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:29:37.292491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624183+0000","last_change":"2024-02-18T11:47:34.546010+0000","last_active":"2024-02-18T11:47:39.624183+0000","last_peered":"2024-02-18T11:47:39.624183+0000","last_clean":"2024-02-18T11:47:39.624183+0000","last_became_active":"2024-02-18T11:47:34.545247+0000","last_became_peered":"2024-02-18T11:47:34.545247+0000","last_unstale":"2024-02-18T11:47:39.624183+0000","last_undegraded":"2024-02-18T11:47:39.624183+0000","last_fullsized":"2024-02-18T11:47:39.624183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555993+0000","last_change":"2024-02-18T11:47:34.538436+0000","last_active":"2024-02-18T11:47:39.555993+0000","last_peered":"2024-02-18T11:47:39.555993+0000","last_clean":"2024-02-18T11:47:39.555993+0000","last_became_active":"2024-02-18T11:47:34.538207+0000","last_became_peered":"2024-02-18T11:47:34.538207+0000","last_unstale":"2024-02-18T11:47:39.555993+0000","last_undegraded":"2024-02-18T11:47:39.555993+0000","last_fullsized":"2024-02-18T11:47:39.555993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845187+0000","last_change":"2024-02-18T11:47:34.542741+0000","last_active":"2024-02-18T11:47:39.845187+0000","last_peered":"2024-02-18T11:47:39.845187+0000","last_clean":"2024-02-18T11:47:39.845187+0000","last_became_active":"2024-02-18T11:47:34.542467+0000","last_became_peered":"2024-02-18T11:47:34.542467+0000","last_unstale":"2024-02-18T11:47:39.845187+0000","last_undegraded":"2024-02-18T11:47:39.845187+0000","last_fullsized":"2024-02-18T11:47:39.845187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557105+0000","last_change":"2024-02-18T11:47:34.539970+0000","last_active":"2024-02-18T11:47:39.557105+0000","last_peered":"2024-02-18T11:47:39.557105+0000","last_clean":"2024-02-18T11:47:39.557105+0000","last_became_active":"2024-02-18T11:47:34.539471+0000","last_became_peered":"2024-02-18T11:47:34.539471+0000","last_unstale":"2024-02-18T11:47:39.557105+0000","last_undegraded":"2024-02-18T11:47:39.557105+0000","last_fullsized":"2024-02-18T11:47:39.557105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555437+0000","last_change":"2024-02-18T11:47:34.536619+0000","last_active":"2024-02-18T11:47:39.555437+0000","last_peered":"2024-02-18T11:47:39.555437+0000","last_clean":"2024-02-18T11:47:39.555437+0000","last_became_active":"2024-02-18T11:47:34.536356+0000","last_became_peered":"2024-02-18T11:47:34.536356+0000","last_unstale":"2024-02-18T11:47:39.555437+0000","last_undegraded":"2024-02-18T11:47:39.555437+0000","last_fullsized":"2024-02-18T11:47:39.555437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:58:25.933027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171840+0000","last_change":"2024-02-18T11:47:34.542717+0000","last_active":"2024-02-18T11:47:40.171840+0000","last_peered":"2024-02-18T11:47:40.171840+0000","last_clean":"2024-02-18T11:47:40.171840+0000","last_became_active":"2024-02-18T11:47:34.542470+0000","last_became_peered":"2024-02-18T11:47:34.542470+0000","last_unstale":"2024-02-18T11:47:40.171840+0000","last_undegraded":"2024-02-18T11:47:40.171840+0000","last_fullsized":"2024-02-18T11:47:40.171840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555193+0000","last_change":"2024-02-18T11:47:34.537727+0000","last_active":"2024-02-18T11:47:39.555193+0000","last_peered":"2024-02-18T11:47:39.555193+0000","last_clean":"2024-02-18T11:47:39.555193+0000","last_became_active":"2024-02-18T11:47:34.536902+0000","last_became_peered":"2024-02-18T11:47:34.536902+0000","last_unstale":"2024-02-18T11:47:39.555193+0000","last_undegraded":"2024-02-18T11:47:39.555193+0000","last_fullsized":"2024-02-18T11:47:39.555193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172026+0000","last_change":"2024-02-18T11:47:34.549313+0000","last_active":"2024-02-18T11:47:40.172026+0000","last_peered":"2024-02-18T11:47:40.172026+0000","last_clean":"2024-02-18T11:47:40.172026+0000","last_became_active":"2024-02-18T11:47:34.549010+0000","last_became_peered":"2024-02-18T11:47:34.549010+0000","last_unstale":"2024-02-18T11:47:40.172026+0000","last_undegraded":"2024-02-18T11:47:40.172026+0000","last_fullsized":"2024-02-18T11:47:40.172026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624061+0000","last_change":"2024-02-18T11:47:34.546238+0000","last_active":"2024-02-18T11:47:39.624061+0000","last_peered":"2024-02-18T11:47:39.624061+0000","last_clean":"2024-02-18T11:47:39.624061+0000","last_became_active":"2024-02-18T11:47:34.545634+0000","last_became_peered":"2024-02-18T11:47:34.545634+0000","last_unstale":"2024-02-18T11:47:39.624061+0000","last_undegraded":"2024-02-18T11:47:39.624061+0000","last_fullsized":"2024-02-18T11:47:39.624061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:18:21.067312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556482+0000","last_change":"2024-02-18T11:47:34.534576+0000","last_active":"2024-02-18T11:47:39.556482+0000","last_peered":"2024-02-18T11:47:39.556482+0000","last_clean":"2024-02-18T11:47:39.556482+0000","last_became_active":"2024-02-18T11:47:34.534232+0000","last_became_peered":"2024-02-18T11:47:34.534232+0000","last_unstale":"2024-02-18T11:47:39.556482+0000","last_undegraded":"2024-02-18T11:47:39.556482+0000","last_fullsized":"2024-02-18T11:47:39.556482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309588+0000","last_change":"2024-02-18T11:47:34.546750+0000","last_active":"2024-02-18T11:47:40.309588+0000","last_peered":"2024-02-18T11:47:40.309588+0000","last_clean":"2024-02-18T11:47:40.309588+0000","last_became_active":"2024-02-18T11:47:34.546507+0000","last_became_peered":"2024-02-18T11:47:34.546507+0000","last_unstale":"2024-02-18T11:47:40.309588+0000","last_undegraded":"2024-02-18T11:47:40.309588+0000","last_fullsized":"2024-02-18T11:47:40.309588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:33:21.864869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555043+0000","last_change":"2024-02-18T11:47:34.542028+0000","last_active":"2024-02-18T11:47:39.555043+0000","last_peered":"2024-02-18T11:47:39.555043+0000","last_clean":"2024-02-18T11:47:39.555043+0000","last_became_active":"2024-02-18T11:47:34.541653+0000","last_became_peered":"2024-02-18T11:47:34.541653+0000","last_unstale":"2024-02-18T11:47:39.555043+0000","last_undegraded":"2024-02-18T11:47:39.555043+0000","last_fullsized":"2024-02-18T11:47:39.555043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:03:38.652128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172499+0000","last_change":"2024-02-18T11:47:34.549989+0000","last_active":"2024-02-18T11:47:40.172499+0000","last_peered":"2024-02-18T11:47:40.172499+0000","last_clean":"2024-02-18T11:47:40.172499+0000","last_became_active":"2024-02-18T11:47:34.549661+0000","last_became_peered":"2024-02-18T11:47:34.549661+0000","last_unstale":"2024-02-18T11:47:40.172499+0000","last_undegraded":"2024-02-18T11:47:40.172499+0000","last_fullsized":"2024-02-18T11:47:40.172499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171732+0000","last_change":"2024-02-18T11:47:34.547081+0000","last_active":"2024-02-18T11:47:40.171732+0000","last_peered":"2024-02-18T11:47:40.171732+0000","last_clean":"2024-02-18T11:47:40.171732+0000","last_became_active":"2024-02-18T11:47:34.546812+0000","last_became_peered":"2024-02-18T11:47:34.546812+0000","last_unstale":"2024-02-18T11:47:40.171732+0000","last_undegraded":"2024-02-18T11:47:40.171732+0000","last_fullsized":"2024-02-18T11:47:40.171732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:18:55.125522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.844848+0000","last_change":"2024-02-18T11:47:34.534907+0000","last_active":"2024-02-18T11:47:39.844848+0000","last_peered":"2024-02-18T11:47:39.844848+0000","last_clean":"2024-02-18T11:47:39.844848+0000","last_became_active":"2024-02-18T11:47:34.534661+0000","last_became_peered":"2024-02-18T11:47:34.534661+0000","last_unstale":"2024-02-18T11:47:39.844848+0000","last_undegraded":"2024-02-18T11:47:39.844848+0000","last_fullsized":"2024-02-18T11:47:39.844848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624050+0000","last_change":"2024-02-18T11:47:34.537888+0000","last_active":"2024-02-18T11:47:39.624050+0000","last_peered":"2024-02-18T11:47:39.624050+0000","last_clean":"2024-02-18T11:47:39.624050+0000","last_became_active":"2024-02-18T11:47:34.537647+0000","last_became_peered":"2024-02-18T11:47:34.537647+0000","last_unstale":"2024-02-18T11:47:39.624050+0000","last_undegraded":"2024-02-18T11:47:39.624050+0000","last_fullsized":"2024-02-18T11:47:39.624050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:40:02.426785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555401+0000","last_change":"2024-02-18T11:47:34.541254+0000","last_active":"2024-02-18T11:47:39.555401+0000","last_peered":"2024-02-18T11:47:39.555401+0000","last_clean":"2024-02-18T11:47:39.555401+0000","last_became_active":"2024-02-18T11:47:34.540955+0000","last_became_peered":"2024-02-18T11:47:34.540955+0000","last_unstale":"2024-02-18T11:47:39.555401+0000","last_undegraded":"2024-02-18T11:47:39.555401+0000","last_fullsized":"2024-02-18T11:47:39.555401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.808837+0000","last_change":"2024-02-18T11:47:34.531217+0000","last_active":"2024-02-18T11:47:39.808837+0000","last_peered":"2024-02-18T11:47:39.808837+0000","last_clean":"2024-02-18T11:47:39.808837+0000","last_became_active":"2024-02-18T11:47:34.530120+0000","last_became_peered":"2024-02-18T11:47:34.530120+0000","last_unstale":"2024-02-18T11:47:39.808837+0000","last_undegraded":"2024-02-18T11:47:39.808837+0000","last_fullsized":"2024-02-18T11:47:39.808837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:08:43.018162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.556645+0000","last_change":"2024-02-18T11:47:34.544075+0000","last_active":"2024-02-18T11:47:39.556645+0000","last_peered":"2024-02-18T11:47:39.556645+0000","last_clean":"2024-02-18T11:47:39.556645+0000","last_became_active":"2024-02-18T11:47:34.543761+0000","last_became_peered":"2024-02-18T11:47:34.543761+0000","last_unstale":"2024-02-18T11:47:39.556645+0000","last_undegraded":"2024-02-18T11:47:39.556645+0000","last_fullsized":"2024-02-18T11:47:39.556645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:30:59.305627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.557031+0000","last_change":"2024-02-18T11:47:34.546387+0000","last_active":"2024-02-18T11:47:39.557031+0000","last_peered":"2024-02-18T11:47:39.557031+0000","last_clean":"2024-02-18T11:47:39.557031+0000","last_became_active":"2024-02-18T11:47:34.546115+0000","last_became_peered":"2024-02-18T11:47:34.546115+0000","last_unstale":"2024-02-18T11:47:39.557031+0000","last_undegraded":"2024-02-18T11:47:39.557031+0000","last_fullsized":"2024-02-18T11:47:39.557031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309697+0000","last_change":"2024-02-18T11:47:34.552880+0000","last_active":"2024-02-18T11:47:40.309697+0000","last_peered":"2024-02-18T11:47:40.309697+0000","last_clean":"2024-02-18T11:47:40.309697+0000","last_became_active":"2024-02-18T11:47:34.552530+0000","last_became_peered":"2024-02-18T11:47:34.552530+0000","last_unstale":"2024-02-18T11:47:40.309697+0000","last_undegraded":"2024-02-18T11:47:40.309697+0000","last_fullsized":"2024-02-18T11:47:40.309697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:16:00.069929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.555106+0000","last_change":"2024-02-18T11:47:34.545727+0000","last_active":"2024-02-18T11:47:39.555106+0000","last_peered":"2024-02-18T11:47:39.555106+0000","last_clean":"2024-02-18T11:47:39.555106+0000","last_became_active":"2024-02-18T11:47:34.545475+0000","last_became_peered":"2024-02-18T11:47:34.545475+0000","last_unstale":"2024-02-18T11:47:39.555106+0000","last_undegraded":"2024-02-18T11:47:39.555106+0000","last_fullsized":"2024-02-18T11:47:39.555106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:43:38.952905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309012+0000","last_change":"2024-02-18T11:47:34.547696+0000","last_active":"2024-02-18T11:47:40.309012+0000","last_peered":"2024-02-18T11:47:40.309012+0000","last_clean":"2024-02-18T11:47:40.309012+0000","last_became_active":"2024-02-18T11:47:34.547448+0000","last_became_peered":"2024-02-18T11:47:34.547448+0000","last_unstale":"2024-02-18T11:47:40.309012+0000","last_undegraded":"2024-02-18T11:47:40.309012+0000","last_fullsized":"2024-02-18T11:47:40.309012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:07:32.750265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.171835+0000","last_change":"2024-02-18T11:47:34.548053+0000","last_active":"2024-02-18T11:47:40.171835+0000","last_peered":"2024-02-18T11:47:40.171835+0000","last_clean":"2024-02-18T11:47:40.171835+0000","last_became_active":"2024-02-18T11:47:34.547181+0000","last_became_peered":"2024-02-18T11:47:34.547181+0000","last_unstale":"2024-02-18T11:47:40.171835+0000","last_undegraded":"2024-02-18T11:47:40.171835+0000","last_fullsized":"2024-02-18T11:47:40.171835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:25:47.171727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309134+0000","last_change":"2024-02-18T11:47:34.549242+0000","last_active":"2024-02-18T11:47:40.309134+0000","last_peered":"2024-02-18T11:47:40.309134+0000","last_clean":"2024-02-18T11:47:40.309134+0000","last_became_active":"2024-02-18T11:47:34.548872+0000","last_became_peered":"2024-02-18T11:47:34.548872+0000","last_unstale":"2024-02-18T11:47:40.309134+0000","last_undegraded":"2024-02-18T11:47:40.309134+0000","last_fullsized":"2024-02-18T11:47:40.309134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:59:43.779748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.172083+0000","last_change":"2024-02-18T11:47:34.546466+0000","last_active":"2024-02-18T11:47:40.172083+0000","last_peered":"2024-02-18T11:47:40.172083+0000","last_clean":"2024-02-18T11:47:40.172083+0000","last_became_active":"2024-02-18T11:47:34.546218+0000","last_became_peered":"2024-02-18T11:47:34.546218+0000","last_unstale":"2024-02-18T11:47:40.172083+0000","last_undegraded":"2024-02-18T11:47:40.172083+0000","last_fullsized":"2024-02-18T11:47:40.172083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:57:55.404808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.624360+0000","last_change":"2024-02-18T11:47:34.530060+0000","last_active":"2024-02-18T11:47:39.624360+0000","last_peered":"2024-02-18T11:47:39.624360+0000","last_clean":"2024-02-18T11:47:39.624360+0000","last_became_active":"2024-02-18T11:47:34.529799+0000","last_became_peered":"2024-02-18T11:47:34.529799+0000","last_unstale":"2024-02-18T11:47:39.624360+0000","last_undegraded":"2024-02-18T11:47:39.624360+0000","last_fullsized":"2024-02-18T11:47:39.624360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:41:39.029754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:39.845225+0000","last_change":"2024-02-18T11:47:34.547273+0000","last_active":"2024-02-18T11:47:39.845225+0000","last_peered":"2024-02-18T11:47:39.845225+0000","last_clean":"2024-02-18T11:47:39.845225+0000","last_became_active":"2024-02-18T11:47:34.546984+0000","last_became_peered":"2024-02-18T11:47:34.546984+0000","last_unstale":"2024-02-18T11:47:39.845225+0000","last_undegraded":"2024-02-18T11:47:39.845225+0000","last_fullsized":"2024-02-18T11:47:39.845225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:18:07.030360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-02-18T11:47:40.309602+0000","last_change":"2024-02-18T11:47:34.547456+0000","last_active":"2024-02-18T11:47:40.309602+0000","last_peered":"2024-02-18T11:47:40.309602+0000","last_clean":"2024-02-18T11:47:40.309602+0000","last_became_active":"2024-02-18T11:47:34.547193+0000","last_became_peered":"2024-02-18T11:47:34.547193+0000","last_unstale":"2024-02-18T11:47:40.309602+0000","last_undegraded":"2024-02-18T11:47:40.309602+0000","last_fullsized":"2024-02-18T11:47:40.309602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:33.499472+0000","last_clean_scrub_stamp":"2024-02-18T11:47:33.499472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:23:05.746813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:02.212 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-02-18T11:48:02.212 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-18T11:48:02.212 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-02-18T11:48:02.212 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-02-18T11:48:02.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.353+0000 7fdbbb377640 1 Processor -- start 2024-02-18T11:48:02.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.353+0000 7fdbbb377640 1 -- start start 2024-02-18T11:48:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbbb377640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb415ac40 0x7fdbb406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbbb377640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40b02e0 con 0x7fdbb415a860 2024-02-18T11:48:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbbb377640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40af6d0 con 0x7fdbb406e850 2024-02-18T11:48:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbbb377640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb406ece0 con 0x7fdbb415ac40 2024-02-18T11:48:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbb9113640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdbb415a860 0x7fdbb40cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46056/0 (socket says 172.21.15.116:46056) 2024-02-18T11:48:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbb9113640 1 -- 172.21.15.116:0/2321544921 learned_addr learned my addr 172.21.15.116:0/2321544921 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.354+0000 7fdbb8912640 1 --2- 172.21.15.116:0/2321544921 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb415ac40 0x7fdbb406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117159284 0 0) 0x7fdbb40b02e0 con 0x7fdbb415a860 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdba0003520 con 0x7fdbb415a860 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547557520 0 0) 0x7fdbb40af6d0 con 0x7fdbb406e850 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb40b02e0 con 0x7fdbb406e850 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbb8912640 1 -- 172.21.15.116:0/2321544921 >> v1:172.21.15.143:6789/0 conn(0x7fdbb406e850 legacy=0x7fdbb40b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbb8912640 1 -- 172.21.15.116:0/2321544921 >> v1:172.21.15.116:6789/0 conn(0x7fdbb415a860 legacy=0x7fdbb40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3284767803 0 0) 0x7fdba0003520 con 0x7fdbb415a860 2024-02-18T11:48:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.355+0000 7fdbb8912640 1 -- 172.21.15.116:0/2321544921 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb40af9b0 con 0x7fdbb415ac40 2024-02-18T11:48:02.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbb8912640 1 --2- 172.21.15.116:0/2321544921 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb415ac40 0x7fdbb406e310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdba40097d0 tx=0x7fdba402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb9788dec50f4eb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00019f0 con 0x7fdbb415ac40 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba402e050 con 0x7fdbb415ac40 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbabfff640 1 -- 172.21.15.116:0/2321544921 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba4030710 con 0x7fdbb415ac40 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 -- 172.21.15.116:0/2321544921 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb415ac40 msgr2=0x7fdbb406e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 --2- 172.21.15.116:0/2321544921 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb415ac40 0x7fdbb406e310 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdba40097d0 tx=0x7fdba402b800 comp rx=0 tx=0).stop 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 -- 172.21.15.116:0/2321544921 shutdown_connections 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 --2- 172.21.15.116:0/2321544921 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb415ac40 0x7fdbb406e310 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 -- 172.21.15.116:0/2321544921 >> 172.21.15.116:0/2321544921 conn(0x7fdbb4066f20 msgr2=0x7fdbb406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 -- 172.21.15.116:0/2321544921 shutdown_connections 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.356+0000 7fdbbb377640 1 -- 172.21.15.116:0/2321544921 wait complete. 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.357+0000 7fdbbb377640 1 Processor -- start 2024-02-18T11:48:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 -- start start 2024-02-18T11:48:02.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb406e850 0x7fdbb41f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 0x7fdbb41eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb415ac40 0x7fdbb41550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb414ea20 con 0x7fdbb415a860 2024-02-18T11:48:02.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4154910 con 0x7fdbb415ac40 2024-02-18T11:48:02.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbbb377640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41ef770 con 0x7fdbb406e850 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbb8912640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 0x7fdbb41eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbb8912640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 0x7fdbb41eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57584/0 (socket says 172.21.15.116:57584) 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.358+0000 7fdbb8912640 1 -- 172.21.15.116:0/3100909376 learned_addr learned my addr 172.21.15.116:0/3100909376 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb9113640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb406e850 0x7fdbb41f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb8912640 1 -- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb406e850 msgr2=0x7fdbb41f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb8912640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb406e850 0x7fdbb41f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb8912640 1 -- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb415ac40 msgr2=0x7fdbb41550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb9914640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb415ac40 0x7fdbb41550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb8912640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb415ac40 0x7fdbb41550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb8912640 1 -- 172.21.15.116:0/3100909376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4009400 con 0x7fdbb415a860 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbb8912640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 0x7fdbb41eef40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdba402b840 tx=0x7fdba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba403b810 con 0x7fdbb415a860 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba403b970 con 0x7fdbb415a860 2024-02-18T11:48:02.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb41efa50 con 0x7fdbb415a860 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba403bd00 con 0x7fdbb415a860 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.359+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb4154bb0 con 0x7fdbb415a860 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.360+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb4074520 con 0x7fdbb415a860 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.361+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fdba4039040 con 0x7fdbb415a860 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.361+0000 7fdba9ffb640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdb9806bc70 0x7fdb9806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.361+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdba40ac260 con 0x7fdbb415a860 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.361+0000 7fdbb9113640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdb9806bc70 0x7fdb9806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.361+0000 7fdbb9113640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdb9806bc70 0x7fdb9806e130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001460 tx=0x7fdbb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.365+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdba4076700 con 0x7fdbb415a860 2024-02-18T11:48:02.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.575+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fdbb40b3070 con 0x7fdbb415a860 2024-02-18T11:48:02.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.576+0000 7fdba9ffb640 1 -- 172.21.15.116:0/3100909376 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fdba403be60 con 0x7fdbb415a860 2024-02-18T11:48:02.569 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:02.569 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-18T11:48:02.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdb9806bc70 msgr2=0x7fdb9806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdb9806bc70 0x7fdb9806e130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0001460 tx=0x7fdbb0009040 comp rx=0 tx=0).stop 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 msgr2=0x7fdbb41eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 0x7fdbb41eef40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdba402b840 tx=0x7fdba40046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 shutdown_connections 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdb9806bc70 0x7fdb9806e130 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbb415ac40 0x7fdbb41550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbb415a860 0x7fdbb41eef40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 --2- 172.21.15.116:0/3100909376 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdbb406e850 0x7fdbb41f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 >> 172.21.15.116:0/3100909376 conn(0x7fdbb4066f20 msgr2=0x7fdbb40b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 shutdown_connections 2024-02-18T11:48:02.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.578+0000 7fdbbb377640 1 -- 172.21.15.116:0/3100909376 wait complete. 2024-02-18T11:48:02.583 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-02-18T11:48:02.583 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:02.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.690+0000 7f6ef0a40640 1 Processor -- start 2024-02-18T11:48:02.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.690+0000 7f6ef0a40640 1 -- start start 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6ef0a40640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6eec0d0b90 0x7f6eec0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6ef0a40640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec0ccb70 con 0x7f6eec0b0750 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6ef0a40640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec0cf970 con 0x7f6eec0d0730 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6ef0a40640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0cfc50 con 0x7f6eec0d0b90 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6eead9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6eec0d0b90 0x7f6eec0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6eea59c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6eec0b0750 0x7f6eec0b0b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46072/0 (socket says 172.21.15.116:46072) 2024-02-18T11:48:02.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6eea59c640 1 -- 172.21.15.116:0/1681763953 learned_addr learned my addr 172.21.15.116:0/1681763953 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.691+0000 7f6eead9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6eec0d0b90 0x7f6eec0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57586/0 (socket says 172.21.15.116:57586) 2024-02-18T11:48:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.692+0000 7f6eead9d640 1 -- 172.21.15.116:0/1681763953 >> v1:172.21.15.143:6789/0 conn(0x7f6eec0d0730 legacy=0x7f6eec0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.692+0000 7f6eead9d640 1 -- 172.21.15.116:0/1681763953 >> v1:172.21.15.116:6789/0 conn(0x7f6eec0b0750 legacy=0x7f6eec0b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.692+0000 7f6ee959a640 1 -- 172.21.15.116:0/1681763953 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894819997 0 0) 0x7f6eec0ccb70 con 0x7f6eec0b0750 2024-02-18T11:48:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.692+0000 7f6eead9d640 1 -- 172.21.15.116:0/1681763953 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0cff30 con 0x7f6eec0d0b90 2024-02-18T11:48:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.692+0000 7f6eead9d640 1 --2- 172.21.15.116:0/1681763953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0b90 0x7f6eec0cc550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40097a0 tx=0x7f6ee402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dd1e2c1a5ea580 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ee959a640 1 -- 172.21.15.116:0/1681763953 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4031830 con 0x7f6eec0d0b90 2024-02-18T11:48:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ee959a640 1 -- 172.21.15.116:0/1681763953 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee4031990 con 0x7f6eec0d0b90 2024-02-18T11:48:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ee959a640 1 -- 172.21.15.116:0/1681763953 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4031d20 con 0x7f6eec0d0b90 2024-02-18T11:48:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 -- 172.21.15.116:0/1681763953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0b90 msgr2=0x7f6eec0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/1681763953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0b90 0x7f6eec0cc550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40097a0 tx=0x7f6ee402b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 -- 172.21.15.116:0/1681763953 shutdown_connections 2024-02-18T11:48:02.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/1681763953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0b90 0x7f6eec0cc550 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 -- 172.21.15.116:0/1681763953 >> 172.21.15.116:0/1681763953 conn(0x7f6eec0b3380 msgr2=0x7f6eec0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 -- 172.21.15.116:0/1681763953 shutdown_connections 2024-02-18T11:48:02.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.693+0000 7f6ef0a40640 1 -- 172.21.15.116:0/1681763953 wait complete. 2024-02-18T11:48:02.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.694+0000 7f6ef0a40640 1 Processor -- start 2024-02-18T11:48:02.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 -- start start 2024-02-18T11:48:02.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eec0b0750 0x7f6eec151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 0x7f6eec152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6eec1509c0 0x7f6eec0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec15d3c0 con 0x7f6eec0d0730 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec152690 con 0x7f6eec0b0750 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ef0a40640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec152910 con 0x7f6eec1509c0 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ee9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 0x7f6eec152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ee9d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 0x7f6eec152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57588/0 (socket says 172.21.15.116:57588) 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6ee9d9b640 1 -- 172.21.15.116:0/3058605268 learned_addr learned my addr 172.21.15.116:0/3058605268 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6eea59c640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eec0b0750 0x7f6eec151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.695+0000 7f6eead9d640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6eec1509c0 0x7f6eec0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ee9d9b640 1 -- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6eec1509c0 msgr2=0x7f6eec0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ee9d9b640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6eec1509c0 0x7f6eec0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ee9d9b640 1 -- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eec0b0750 msgr2=0x7f6eec151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ee9d9b640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eec0b0750 0x7f6eec151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ee9d9b640 1 -- 172.21.15.116:0/3058605268 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4009400 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6eead9d640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6eec1509c0 0x7f6eec0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ee9d9b640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 0x7f6eec152150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c460 tx=0x7f6ee000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0012830 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee00129d0 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec152bf0 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0012d80 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.696+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec15d8f0 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.697+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec15db10 con 0x7f6eec0d0730 2024-02-18T11:48:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.697+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f6ee001b400 con 0x7f6eec0d0730 2024-02-18T11:48:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.698+0000 7f6edaffd640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6ecc06bca0 0x7f6ecc06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.698+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ee008a9a0 con 0x7f6eec0d0730 2024-02-18T11:48:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.698+0000 7f6eea59c640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6ecc06bca0 0x7f6ecc06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.698+0000 7f6eea59c640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6ecc06bca0 0x7f6ecc06e160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0012e0 tx=0x7f6edc001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.702+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ee0054e40 con 0x7f6eec0d0730 2024-02-18T11:48:02.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.874+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6eec15a5f0 con 0x7f6eec0d0730 2024-02-18T11:48:02.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.876+0000 7f6edaffd640 1 -- 172.21.15.116:0/3058605268 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f6ee0058500 con 0x7f6eec0d0730 2024-02-18T11:48:02.869 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:02.869 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.877+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6ecc06bca0 msgr2=0x7f6ecc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.877+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6ecc06bca0 0x7f6ecc06e160 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0012e0 tx=0x7f6edc001700 comp rx=0 tx=0).stop 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.877+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 msgr2=0x7f6eec152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.877+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 0x7f6eec152150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c460 tx=0x7f6ee000c920 comp rx=0 tx=0).stop 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 shutdown_connections 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6ecc06bca0 0x7f6ecc06e160 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6eec1509c0 0x7f6eec0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eec0d0730 0x7f6eec152150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 --2- 172.21.15.116:0/3058605268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6eec0b0750 0x7f6eec151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 >> 172.21.15.116:0/3058605268 conn(0x7f6eec0b3380 msgr2=0x7f6eec0b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 shutdown_connections 2024-02-18T11:48:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.878+0000 7f6ef0a40640 1 -- 172.21.15.116:0/3058605268 wait complete. 2024-02-18T11:48:02.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:02.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.992+0000 7fa94672c640 1 Processor -- start 2024-02-18T11:48:02.986 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.992+0000 7fa94672c640 1 -- start start 2024-02-18T11:48:02.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa94672c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa94015a870 0x7fa9400b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa94672c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9400b4190 0x7fa9400d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa94672c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9400d1020 con 0x7fa940070260 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa94672c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400d0b30 con 0x7fa94015a870 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa94672c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400b5190 con 0x7fa9400b4190 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa93ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa94015a870 0x7fa9400b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa93ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa94015a870 0x7fa9400b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57600/0 (socket says 172.21.15.116:57600) 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa93ffff640 1 -- 172.21.15.116:0/2345683086 learned_addr learned my addr 172.21.15.116:0/2345683086 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.993+0000 7fa944cc9640 1 --1- 172.21.15.116:0/2345683086 >> v1:172.21.15.116:6789/0 conn(0x7fa940070260 0x7fa9400ce610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46084/0 (socket says 172.21.15.116:46084) 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93f7fe640 1 --2- 172.21.15.116:0/2345683086 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9400b4190 0x7fa9400d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93ffff640 1 -- 172.21.15.116:0/2345683086 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9400b4190 msgr2=0x7fa9400d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93ffff640 1 --2- 172.21.15.116:0/2345683086 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9400b4190 0x7fa9400d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93ffff640 1 -- 172.21.15.116:0/2345683086 >> v1:172.21.15.116:6789/0 conn(0x7fa940070260 legacy=0x7fa9400ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93ffff640 1 -- 172.21.15.116:0/2345683086 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400b45f0 con 0x7fa94015a870 2024-02-18T11:48:02.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93effd640 1 -- 172.21.15.116:0/2345683086 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964770990 0 0) 0x7fa9400d1020 con 0x7fa940070260 2024-02-18T11:48:02.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93ffff640 1 --2- 172.21.15.116:0/2345683086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94015a870 0x7fa9400b5620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa93000bb10 tx=0x7fa93002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0328722a9c56fa server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.994+0000 7fa93effd640 1 -- 172.21.15.116:0/2345683086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93002ede0 con 0x7fa94015a870 2024-02-18T11:48:02.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa93effd640 1 -- 172.21.15.116:0/2345683086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930036440 con 0x7fa94015a870 2024-02-18T11:48:02.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa93effd640 1 -- 172.21.15.116:0/2345683086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9300367d0 con 0x7fa94015a870 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 -- 172.21.15.116:0/2345683086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94015a870 msgr2=0x7fa9400b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 --2- 172.21.15.116:0/2345683086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94015a870 0x7fa9400b5620 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa93000bb10 tx=0x7fa93002e030 comp rx=0 tx=0).stop 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 -- 172.21.15.116:0/2345683086 shutdown_connections 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 --2- 172.21.15.116:0/2345683086 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9400b4190 0x7fa9400d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 --2- 172.21.15.116:0/2345683086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94015a870 0x7fa9400b5620 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 -- 172.21.15.116:0/2345683086 >> 172.21.15.116:0/2345683086 conn(0x7fa9400b23f0 msgr2=0x7fa9400b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 -- 172.21.15.116:0/2345683086 shutdown_connections 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.995+0000 7fa94672c640 1 -- 172.21.15.116:0/2345683086 wait complete. 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.996+0000 7fa94672c640 1 Processor -- start 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.996+0000 7fa94672c640 1 -- start start 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa94672c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa940070260 0x7fa940152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa94672c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 0x7fa94015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa94672c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa94015a870 0x7fa940158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa94672c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9401f3860 con 0x7fa9400b4190 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa94672c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400ae990 con 0x7fa940070260 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa94672c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400aec70 con 0x7fa94015a870 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa93f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 0x7fa94015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa93f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 0x7fa94015bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57608/0 (socket says 172.21.15.116:57608) 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa93f7fe640 1 -- 172.21.15.116:0/3024551649 learned_addr learned my addr 172.21.15.116:0/3024551649 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa944cc9640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa94015a870 0x7fa940158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.997+0000 7fa93ffff640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa940070260 0x7fa940152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93f7fe640 1 -- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa94015a870 msgr2=0x7fa940158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93f7fe640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa94015a870 0x7fa940158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93f7fe640 1 -- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa940070260 msgr2=0x7fa940152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93f7fe640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa940070260 0x7fa940152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93f7fe640 1 -- 172.21.15.116:0/3024551649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93000b770 con 0x7fa9400b4190 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93f7fe640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 0x7fa94015bf40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa93400d800 tx=0x7fa93400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400b6c0 con 0x7fa9400b4190 2024-02-18T11:48:02.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93400b820 con 0x7fa9400b4190 2024-02-18T11:48:02.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93400bbb0 con 0x7fa9400b4190 2024-02-18T11:48:02.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9400aef50 con 0x7fa9400b4190 2024-02-18T11:48:02.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.998+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa940152f50 con 0x7fa9400b4190 2024-02-18T11:48:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.999+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9400af470 con 0x7fa9400b4190 2024-02-18T11:48:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:02.999+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa93400bd10 con 0x7fa9400b4190 2024-02-18T11:48:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.000+0000 7fa93cff9640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa92c06bca0 0x7fa92c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.000+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa93408af20 con 0x7fa9400b4190 2024-02-18T11:48:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.000+0000 7fa93ffff640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa92c06bca0 0x7fa92c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.000+0000 7fa93ffff640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa92c06bca0 0x7fa92c06e160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa930038000 tx=0x7fa930032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:02.997 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.004+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9340553e0 con 0x7fa9400b4190 2024-02-18T11:48:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.175+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa940074520 con 0x7fa9400b4190 2024-02-18T11:48:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.176+0000 7fa93cff9640 1 -- 172.21.15.116:0/3024551649 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa934058aa0 con 0x7fa9400b4190 2024-02-18T11:48:03.169 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa92c06bca0 msgr2=0x7fa92c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa92c06bca0 0x7fa92c06e160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa930038000 tx=0x7fa930032000 comp rx=0 tx=0).stop 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 msgr2=0x7fa94015bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 0x7fa94015bf40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa93400d800 tx=0x7fa93400dcc0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 shutdown_connections 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa92c06bca0 0x7fa92c06e160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa94015a870 0x7fa940158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9400b4190 0x7fa94015bf40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 --2- 172.21.15.116:0/3024551649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa940070260 0x7fa940152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 >> 172.21.15.116:0/3024551649 conn(0x7fa9400b23f0 msgr2=0x7fa940155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 shutdown_connections 2024-02-18T11:48:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.178+0000 7fa94672c640 1 -- 172.21.15.116:0/3024551649 wait complete. 2024-02-18T11:48:03.182 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:03.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.292+0000 7f983e8c9640 1 Processor -- start 2024-02-18T11:48:03.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.292+0000 7f983e8c9640 1 -- start start 2024-02-18T11:48:03.286 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983e8c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f983815dbf0 0x7f98381617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983e8c9640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98380b9370 0x7f98380ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983e8c9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98380cec80 0x7f983806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983e8c9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380cf110 con 0x7f98380cec80 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983e8c9640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380d1ee0 con 0x7f98380b9370 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983e8c9640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9838161dc0 con 0x7f983815dbf0 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98380cec80 0x7f983806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98380cec80 0x7f983806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57622/0 (socket says 172.21.15.116:57622) 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 -- 172.21.15.116:0/3174396252 learned_addr learned my addr 172.21.15.116:0/3174396252 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f98377fe640 1 --2- 172.21.15.116:0/3174396252 >> v2:172.21.15.143:3300/0 conn(0x7f98380b9370 0x7f98380ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f9837fff640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f983815dbf0 0x7f98381617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 -- 172.21.15.116:0/3174396252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f983815dbf0 msgr2=0x7f98381617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f983815dbf0 0x7f98381617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 -- 172.21.15.116:0/3174396252 >> v2:172.21.15.143:3300/0 conn(0x7f98380b9370 msgr2=0x7f98380ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 --2- 172.21.15.116:0/3174396252 >> v2:172.21.15.143:3300/0 conn(0x7f98380b9370 0x7f98380ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f983ce66640 1 -- 172.21.15.116:0/3174396252 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98381620a0 con 0x7f98380cec80 2024-02-18T11:48:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f98377fe640 1 --2- 172.21.15.116:0/3174396252 >> v2:172.21.15.143:3300/0 conn(0x7f98380b9370 0x7f98380ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.293+0000 7f9837fff640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f983815dbf0 0x7f98381617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.294+0000 7f983ce66640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f983806c210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9838068310 tx=0x7f983002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28dd690b1cebb9c server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.294+0000 7f9836ffd640 1 -- 172.21.15.116:0/3174396252 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983002ca70 con 0x7f98380cec80 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.294+0000 7f9836ffd640 1 -- 172.21.15.116:0/3174396252 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983002cbd0 con 0x7f98380cec80 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.294+0000 7f9836ffd640 1 -- 172.21.15.116:0/3174396252 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300343f0 con 0x7f98380cec80 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.294+0000 7f983e8c9640 1 -- 172.21.15.116:0/3174396252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 msgr2=0x7f983806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.294+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f983806c210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9838068310 tx=0x7f983002bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 -- 172.21.15.116:0/3174396252 shutdown_connections 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f983806c210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3174396252 >> v2:172.21.15.143:3300/0 conn(0x7f98380b9370 0x7f98380ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3174396252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f983815dbf0 0x7f98381617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 -- 172.21.15.116:0/3174396252 >> 172.21.15.116:0/3174396252 conn(0x7f98380b5480 msgr2=0x7f98380b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 -- 172.21.15.116:0/3174396252 shutdown_connections 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.295+0000 7f983e8c9640 1 -- 172.21.15.116:0/3174396252 wait complete. 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.296+0000 7f983e8c9640 1 Processor -- start 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.296+0000 7f983e8c9640 1 -- start start 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98380b9370 0x7f9838151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f9838157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f983815dbf0 0x7f983815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381580f0 con 0x7f98380cec80 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380c89e0 con 0x7f983815dbf0 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9838157b30 con 0x7f98380b9370 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f9838157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f9838157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57634/0 (socket says 172.21.15.116:57634) 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 -- 172.21.15.116:0/3173827585 learned_addr learned my addr 172.21.15.116:0/3173827585 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983ce66640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f983815dbf0 0x7f983815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f9837fff640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98380b9370 0x7f9838151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 -- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98380b9370 msgr2=0x7f9838151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98380b9370 0x7f9838151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 -- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f983815dbf0 msgr2=0x7f983815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f983815dbf0 0x7f983815ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 -- 172.21.15.116:0/3173827585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830009b30 con 0x7f98380cec80 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983ce66640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f983815dbf0 0x7f983815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f98377fe640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f9838157040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f982c00e810 tx=0x7f982c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c004030 con 0x7f98380cec80 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c004190 con 0x7f98380cec80 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983815e150 con 0x7f98380cec80 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c004520 con 0x7f98380cec80 2024-02-18T11:48:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.297+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98380c8670 con 0x7f98380cec80 2024-02-18T11:48:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.298+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838067840 con 0x7f98380cec80 2024-02-18T11:48:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.299+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f982c00c6d0 con 0x7f98380cec80 2024-02-18T11:48:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.299+0000 7f9834ff9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f982406bc70 0x7f982406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.299+0000 7f9837fff640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f982406bc70 0x7f982406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.299+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f982c08a650 con 0x7f98380cec80 2024-02-18T11:48:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.300+0000 7f9837fff640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f982406bc70 0x7f982406e130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98280096d0 tx=0x7f9828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.304+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f982c054af0 con 0x7f98380cec80 2024-02-18T11:48:03.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.476+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98380c84d0 con 0x7f98380cec80 2024-02-18T11:48:03.470 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.477+0000 7f9834ff9640 1 -- 172.21.15.116:0/3173827585 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98380c84d0 con 0x7f98380cec80 2024-02-18T11:48:03.470 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f982406bc70 msgr2=0x7f982406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f982406bc70 0x7f982406e130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98280096d0 tx=0x7f9828008040 comp rx=0 tx=0).stop 2024-02-18T11:48:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 msgr2=0x7f9838157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f9838157040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f982c00e810 tx=0x7f982c00ecd0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 shutdown_connections 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f982406bc70 0x7f982406e130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f983815dbf0 0x7f983815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98380cec80 0x7f9838157040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 --2- 172.21.15.116:0/3173827585 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98380b9370 0x7f9838151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 >> 172.21.15.116:0/3173827585 conn(0x7f98380b5480 msgr2=0x7f9838158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 shutdown_connections 2024-02-18T11:48:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.479+0000 7f983e8c9640 1 -- 172.21.15.116:0/3173827585 wait complete. 2024-02-18T11:48:03.484 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:03.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.594+0000 7f9d967a0640 1 Processor -- start 2024-02-18T11:48:03.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.594+0000 7f9d967a0640 1 -- start start 2024-02-18T11:48:03.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d967a0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d900ce110 0x7f9d90161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d967a0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9015e360 con 0x7f9d90158040 2024-02-18T11:48:03.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d967a0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900ceaf0 con 0x7f9d900b5330 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d967a0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900ce910 con 0x7f9d900ce110 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d9579e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9d90158040 0x7f9d9015e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46086/0 (socket says 172.21.15.116:46086) 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d94f9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d900ce110 0x7f9d90161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d9579e640 1 -- 172.21.15.116:0/1575253791 learned_addr learned my addr 172.21.15.116:0/1575253791 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d94f9d640 1 -- 172.21.15.116:0/1575253791 >> v1:172.21.15.143:6789/0 conn(0x7f9d900b5330 legacy=0x7f9d900d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d94f9d640 1 -- 172.21.15.116:0/1575253791 >> v1:172.21.15.116:6789/0 conn(0x7f9d90158040 legacy=0x7f9d9015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.595+0000 7f9d94f9d640 1 -- 172.21.15.116:0/1575253791 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900cedd0 con 0x7f9d900ce110 2024-02-18T11:48:03.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.596+0000 7f9d94f9d640 1 --2- 172.21.15.116:0/1575253791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900ce110 0x7f9d90161ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d800097a0 tx=0x7f9d8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d1e8be58c8319b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.596+0000 7f9d87fff640 1 -- 172.21.15.116:0/1575253791 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80031830 con 0x7f9d900ce110 2024-02-18T11:48:03.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.596+0000 7f9d87fff640 1 -- 172.21.15.116:0/1575253791 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d80031990 con 0x7f9d900ce110 2024-02-18T11:48:03.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.596+0000 7f9d87fff640 1 -- 172.21.15.116:0/1575253791 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80031d20 con 0x7f9d900ce110 2024-02-18T11:48:03.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 -- 172.21.15.116:0/1575253791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900ce110 msgr2=0x7f9d90161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1575253791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900ce110 0x7f9d90161ee0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d800097a0 tx=0x7f9d8002b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 -- 172.21.15.116:0/1575253791 shutdown_connections 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1575253791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900ce110 0x7f9d90161ee0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 -- 172.21.15.116:0/1575253791 >> 172.21.15.116:0/1575253791 conn(0x7f9d900d0510 msgr2=0x7f9d900d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 -- 172.21.15.116:0/1575253791 shutdown_connections 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.597+0000 7f9d967a0640 1 -- 172.21.15.116:0/1575253791 wait complete. 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.598+0000 7f9d967a0640 1 Processor -- start 2024-02-18T11:48:03.591 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.598+0000 7f9d967a0640 1 -- start start 2024-02-18T11:48:03.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 0x7f9d9014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9d900ce110 0x7f9d901f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d90158040 0x7f9d9015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.593 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90154120 con 0x7f9d900b5330 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d901543c0 con 0x7f9d90158040 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d901f70e0 con 0x7f9d900ce110 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 0x7f9d9014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 0x7f9d9014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57660/0 (socket says 172.21.15.116:57660) 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 -- 172.21.15.116:0/1667074144 learned_addr learned my addr 172.21.15.116:0/1667074144 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 -- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9d900ce110 msgr2=0x7f9d901f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9d900ce110 0x7f9d901f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 -- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d90158040 msgr2=0x7f9d9015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d95f9f640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d90158040 0x7f9d9015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d90158040 0x7f9d9015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 -- 172.21.15.116:0/1667074144 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80009400 con 0x7f9d900b5330 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d95f9f640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d90158040 0x7f9d9015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d9579e640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 0x7f9d9014fea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c3c0 tx=0x7f9d8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005ce0 con 0x7f9d900b5330 2024-02-18T11:48:03.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c005e40 con 0x7f9d900b5330 2024-02-18T11:48:03.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d901f7380 con 0x7f9d900b5330 2024-02-18T11:48:03.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.599+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0072f0 con 0x7f9d900b5330 2024-02-18T11:48:03.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.600+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d901f7940 con 0x7f9d900b5330 2024-02-18T11:48:03.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.601+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9d8c007450 con 0x7f9d900b5330 2024-02-18T11:48:03.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.601+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d90067840 con 0x7f9d900b5330 2024-02-18T11:48:03.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.601+0000 7f9d85ffb640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d7406bca0 0x7f9d7406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.601+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9d8c089ff0 con 0x7f9d900b5330 2024-02-18T11:48:03.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.601+0000 7f9d94f9d640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d7406bca0 0x7f9d7406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.601+0000 7f9d94f9d640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d7406bca0 0x7f9d7406e160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d80036000 tx=0x7f9d80030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.605+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d8c0bd8c0 con 0x7f9d900b5330 2024-02-18T11:48:03.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.774+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d901f7ad0 con 0x7f9d900b5330 2024-02-18T11:48:03.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.775+0000 7f9d85ffb640 1 -- 172.21.15.116:0/1667074144 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d8c017020 con 0x7f9d900b5330 2024-02-18T11:48:03.768 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.776+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d7406bca0 msgr2=0x7f9d7406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.776+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d7406bca0 0x7f9d7406e160 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d80036000 tx=0x7f9d80030000 comp rx=0 tx=0).stop 2024-02-18T11:48:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 msgr2=0x7f9d9014fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 0x7f9d9014fea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c3c0 tx=0x7f9d8c00c880 comp rx=0 tx=0).stop 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 shutdown_connections 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d7406bca0 0x7f9d7406e160 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d90158040 0x7f9d9015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9d900ce110 0x7f9d901f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 --2- 172.21.15.116:0/1667074144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d900b5330 0x7f9d9014fea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 >> 172.21.15.116:0/1667074144 conn(0x7f9d900d0510 msgr2=0x7f9d900d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 shutdown_connections 2024-02-18T11:48:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.777+0000 7f9d967a0640 1 -- 172.21.15.116:0/1667074144 wait complete. 2024-02-18T11:48:03.781 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:03.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.889+0000 7f9da6cdf640 1 Processor -- start 2024-02-18T11:48:03.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.889+0000 7f9da6cdf640 1 -- start start 2024-02-18T11:48:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da6cdf640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9da00cd610 0x7f9da00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da6cdf640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9da0161fd0 0x7f9da00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da6cdf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da0099a50 con 0x7f9da009ad40 2024-02-18T11:48:03.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da6cdf640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da0099ca0 con 0x7f9da00cd610 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da6cdf640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da0099f80 con 0x7f9da0161fd0 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9da00cd610 0x7f9da00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9da00cd610 0x7f9da00d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57668/0 (socket says 172.21.15.116:57668) 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 -- 172.21.15.116:0/3049217948 learned_addr learned my addr 172.21.15.116:0/3049217948 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da527c640 1 --2- 172.21.15.116:0/3049217948 >> v2:172.21.15.143:3300/0 conn(0x7f9da0161fd0 0x7f9da00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 -- 172.21.15.116:0/3049217948 >> v2:172.21.15.143:3300/0 conn(0x7f9da0161fd0 msgr2=0x7f9da00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 --2- 172.21.15.116:0/3049217948 >> v2:172.21.15.143:3300/0 conn(0x7f9da0161fd0 0x7f9da00d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 -- 172.21.15.116:0/3049217948 >> v1:172.21.15.143:6789/0 conn(0x7f9da009ad40 legacy=0x7f9da01618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da4a7b640 1 -- 172.21.15.116:0/3049217948 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da009a260 con 0x7f9da00cd610 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.890+0000 7f9da527c640 1 --2- 172.21.15.116:0/3049217948 >> v2:172.21.15.143:3300/0 conn(0x7f9da0161fd0 0x7f9da00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.891+0000 7f9da4a7b640 1 --2- 172.21.15.116:0/3049217948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00d0370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9da00b42d0 tx=0x7f9d9c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918616552e325462 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.891+0000 7f9d977fe640 1 -- 172.21.15.116:0/3049217948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c02f850 con 0x7f9da00cd610 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.891+0000 7f9d977fe640 1 -- 172.21.15.116:0/3049217948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c02f9b0 con 0x7f9da00cd610 2024-02-18T11:48:03.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.891+0000 7f9d977fe640 1 -- 172.21.15.116:0/3049217948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c02fd40 con 0x7f9da00cd610 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3049217948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 msgr2=0x7f9da00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3049217948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00d0370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9da00b42d0 tx=0x7f9d9c02ba30 comp rx=0 tx=0).stop 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3049217948 shutdown_connections 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3049217948 >> v2:172.21.15.143:3300/0 conn(0x7f9da0161fd0 0x7f9da00d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3049217948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00d0370 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3049217948 >> 172.21.15.116:0/3049217948 conn(0x7f9da00b31f0 msgr2=0x7f9da00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:03.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3049217948 shutdown_connections 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.892+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3049217948 wait complete. 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.893+0000 7f9da6cdf640 1 Processor -- start 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.893+0000 7f9da6cdf640 1 -- start start 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9da009ad40 0x7f9da014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9da0161fd0 0x7f9da00c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00b6c60 con 0x7f9da00cd610 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00b60e0 con 0x7f9da009ad40 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da00b63c0 con 0x7f9da0161fd0 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57684/0 (socket says 172.21.15.116:57684) 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 -- 172.21.15.116:0/3919719760 learned_addr learned my addr 172.21.15.116:0/3919719760 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da527c640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9da0161fd0 0x7f9da00c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da4a7b640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9da009ad40 0x7f9da014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 -- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9da0161fd0 msgr2=0x7f9da00c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9da0161fd0 0x7f9da00c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 -- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9da009ad40 msgr2=0x7f9da014e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9da009ad40 0x7f9da014e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 -- 172.21.15.116:0/3919719760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c0094d0 con 0x7f9da00cd610 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d97fff640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00c5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c400 tx=0x7f9d9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900052d0 con 0x7f9da00cd610 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90005450 con 0x7f9da00cd610 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da00b66a0 con 0x7f9da00cd610 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90007260 con 0x7f9da00cd610 2024-02-18T11:48:03.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.894+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da014e680 con 0x7f9da00cd610 2024-02-18T11:48:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.895+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da0067840 con 0x7f9da00cd610 2024-02-18T11:48:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.896+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9d900073c0 con 0x7f9da00cd610 2024-02-18T11:48:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.896+0000 7f9d957fa640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d8c06bca0 0x7f9d8c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.896+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9d90089ec0 con 0x7f9da00cd610 2024-02-18T11:48:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.896+0000 7f9da4a7b640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d8c06bca0 0x7f9d8c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:03.891 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.897+0000 7f9da4a7b640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d8c06bca0 0x7f9d8c06e160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c036000 tx=0x7f9d9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:03.893 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:03.900+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d90054360 con 0x7f9da00cd610 2024-02-18T11:48:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.069+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9da0074520 con 0x7f9da00cd610 2024-02-18T11:48:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.069+0000 7f9d957fa640 1 -- 172.21.15.116:0/3919719760 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9d90057a20 con 0x7f9da00cd610 2024-02-18T11:48:04.063 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:04.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d8c06bca0 msgr2=0x7f9d8c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d8c06bca0 0x7f9d8c06e160 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c036000 tx=0x7f9d9c008030 comp rx=0 tx=0).stop 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 msgr2=0x7f9da00c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00c5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c400 tx=0x7f9d9000c8c0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 shutdown_connections 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9d8c06bca0 0x7f9d8c06e160 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9da0161fd0 0x7f9da00c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9da00cd610 0x7f9da00c5230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 --2- 172.21.15.116:0/3919719760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9da009ad40 0x7f9da014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 >> 172.21.15.116:0/3919719760 conn(0x7f9da00b31f0 msgr2=0x7f9da00b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 shutdown_connections 2024-02-18T11:48:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.071+0000 7f9da6cdf640 1 -- 172.21.15.116:0/3919719760 wait complete. 2024-02-18T11:48:04.075 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:04.105 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:48:04.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:48:04.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.184+0000 7f49d956c640 1 Processor -- start 2024-02-18T11:48:04.178 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.184+0000 7f49d956c640 1 -- start start 2024-02-18T11:48:04.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d956c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d415cde0 0x7f49d415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d956c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d40cc720 con 0x7f49d415db50 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d956c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d415d1c0 con 0x7f49d40cc2a0 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d956c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40cd460 con 0x7f49d415cde0 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d37fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f49d415db50 0x7f49d40cf810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46092/0 (socket says 172.21.15.116:46092) 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d37fe640 1 -- 172.21.15.116:0/2205554122 learned_addr learned my addr 172.21.15.116:0/2205554122 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.185+0000 7f49d2ffd640 1 --2- 172.21.15.116:0/2205554122 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d415cde0 0x7f49d415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485392763 0 0) 0x7f49d40cc720 con 0x7f49d415db50 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49bc003520 con 0x7f49d415db50 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139861315 0 0) 0x7f49d415d1c0 con 0x7f49d40cc2a0 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d40cc720 con 0x7f49d40cc2a0 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d2ffd640 1 -- 172.21.15.116:0/2205554122 >> v1:172.21.15.143:6789/0 conn(0x7f49d40cc2a0 legacy=0x7f49d409a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d2ffd640 1 -- 172.21.15.116:0/2205554122 >> v1:172.21.15.116:6789/0 conn(0x7f49d415db50 legacy=0x7f49d40cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1982143692 0 0) 0x7f49bc003520 con 0x7f49d415db50 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.186+0000 7f49d2ffd640 1 -- 172.21.15.116:0/2205554122 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d4099bf0 con 0x7f49d415cde0 2024-02-18T11:48:04.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d2ffd640 1 --2- 172.21.15.116:0/2205554122 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d415cde0 0x7f49d415d610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f49c80097d0 tx=0x7f49c802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5afe3c0c6670478 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0019f0 con 0x7f49d415cde0 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c802e050 con 0x7f49d415cde0 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d1ffb640 1 -- 172.21.15.116:0/2205554122 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8030710 con 0x7f49d415cde0 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 -- 172.21.15.116:0/2205554122 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d415cde0 msgr2=0x7f49d415d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 --2- 172.21.15.116:0/2205554122 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d415cde0 0x7f49d415d610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f49c80097d0 tx=0x7f49c802b800 comp rx=0 tx=0).stop 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 -- 172.21.15.116:0/2205554122 shutdown_connections 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 --2- 172.21.15.116:0/2205554122 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d415cde0 0x7f49d415d610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 -- 172.21.15.116:0/2205554122 >> 172.21.15.116:0/2205554122 conn(0x7f49d40d1790 msgr2=0x7f49d40d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 -- 172.21.15.116:0/2205554122 shutdown_connections 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.187+0000 7f49d956c640 1 -- 172.21.15.116:0/2205554122 wait complete. 2024-02-18T11:48:04.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 Processor -- start 2024-02-18T11:48:04.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 -- start start 2024-02-18T11:48:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d40cc2a0 0x7f49d406d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 0x7f49d40b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415db50 0x7f49d40cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d406bc70 con 0x7f49d415cde0 2024-02-18T11:48:04.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d415aaf0 con 0x7f49d415db50 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d956c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d41f3600 con 0x7f49d40cc2a0 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 0x7f49d40b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.189+0000 7f49d27fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 0x7f49d40b63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57690/0 (socket says 172.21.15.116:57690) 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 -- 172.21.15.116:0/1418917780 learned_addr learned my addr 172.21.15.116:0/1418917780 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d37fe640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415db50 0x7f49d40cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d2ffd640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d40cc2a0 0x7f49d406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 -- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d40cc2a0 msgr2=0x7f49d406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d40cc2a0 0x7f49d406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 -- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415db50 msgr2=0x7f49d40cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415db50 0x7f49d40cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 -- 172.21.15.116:0/1418917780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c8009400 con 0x7f49d415cde0 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d37fe640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415db50 0x7f49d40cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d27fc640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 0x7f49d40b63f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c490 tx=0x7f49c400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40052d0 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4005430 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d40cf660 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4012870 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.190+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d41f38c0 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.191+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d4067840 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.191+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f49c40129d0 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.192+0000 7f49bb7fe640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f49c006bc70 0x7f49c006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.192+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49c408a6c0 con 0x7f49d415cde0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.192+0000 7f49d2ffd640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f49c006bc70 0x7f49c006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.192+0000 7f49d2ffd640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f49c006bc70 0x7f49c006e130 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034000 tx=0x7f49c8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.197+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49c4054b60 con 0x7f49d415cde0 2024-02-18T11:48:04.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.404+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49d4074520 con 0x7f49d415cde0 2024-02-18T11:48:04.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.404+0000 7f49bb7fe640 1 -- 172.21.15.116:0/1418917780 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f49c4058220 con 0x7f49d415cde0 2024-02-18T11:48:04.398 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:04.398 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:04.398 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f49c006bc70 msgr2=0x7f49c006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f49c006bc70 0x7f49c006e130 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f49c8034000 tx=0x7f49c8008030 comp rx=0 tx=0).stop 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 msgr2=0x7f49d40b63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 0x7f49d40b63f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c490 tx=0x7f49c400c950 comp rx=0 tx=0).stop 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 shutdown_connections 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f49c006bc70 0x7f49c006e130 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415db50 0x7f49d40cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d415cde0 0x7f49d40b63f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 --2- 172.21.15.116:0/1418917780 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f49d40cc2a0 0x7f49d406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 >> 172.21.15.116:0/1418917780 conn(0x7f49d40d1790 msgr2=0x7f49d415bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:04.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 shutdown_connections 2024-02-18T11:48:04.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.406+0000 7f49d956c640 1 -- 172.21.15.116:0/1418917780 wait complete. 2024-02-18T11:48:04.411 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:04.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.559+0000 7f582116b640 1 Processor -- start 2024-02-18T11:48:04.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.559+0000 7f582116b640 1 -- start start 2024-02-18T11:48:04.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f582116b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f582116b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f581c06ab60 0x7f581c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f582116b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c06b500 con 0x7f581c06fa70 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f582116b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0d14a0 con 0x7f581c06ab60 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f582116b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0d0400 con 0x7f581c160e90 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f581c06ab60 0x7f581c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f581c06ab60 0x7f581c06afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57700/0 (socket says 172.21.15.116:57700) 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 -- 172.21.15.116:0/389641058 learned_addr learned my addr 172.21.15.116:0/389641058 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 -- 172.21.15.116:0/389641058 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 msgr2=0x7f581c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 --2- 172.21.15.116:0/389641058 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 -- 172.21.15.116:0/389641058 >> v1:172.21.15.143:6789/0 conn(0x7f581c06fa70 legacy=0x7f581c06fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:04.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.560+0000 7f5813fff640 1 -- 172.21.15.116:0/389641058 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0d06e0 con 0x7f581c06ab60 2024-02-18T11:48:04.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f5813fff640 1 --2- 172.21.15.116:0/389641058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06ab60 0x7f581c06afc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1ecbf26d46dac7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f581a59c640 1 -- 172.21.15.116:0/389641058 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804031640 con 0x7f581c06ab60 2024-02-18T11:48:04.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f581a59c640 1 -- 172.21.15.116:0/389641058 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58040317a0 con 0x7f581c06ab60 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f581a59c640 1 -- 172.21.15.116:0/389641058 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804031b30 con 0x7f581c06ab60 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 -- 172.21.15.116:0/389641058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06ab60 msgr2=0x7f581c06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 --2- 172.21.15.116:0/389641058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06ab60 0x7f581c06afc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5804009870 tx=0x7f580402b8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 -- 172.21.15.116:0/389641058 shutdown_connections 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 --2- 172.21.15.116:0/389641058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06ab60 0x7f581c06afc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 --2- 172.21.15.116:0/389641058 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c0afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 -- 172.21.15.116:0/389641058 >> 172.21.15.116:0/389641058 conn(0x7f581c0b23f0 msgr2=0x7f581c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 -- 172.21.15.116:0/389641058 shutdown_connections 2024-02-18T11:48:04.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.561+0000 7f582116b640 1 -- 172.21.15.116:0/389641058 wait complete. 2024-02-18T11:48:04.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 Processor -- start 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 -- start start 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c06ab60 0x7f581c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 0x7f581c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c1519b0 con 0x7f581c06fa70 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c1571d0 con 0x7f581c06ab60 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.563+0000 7f582116b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c151030 con 0x7f581c160e90 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 0x7f581c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 0x7f581c1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57706/0 (socket says 172.21.15.116:57706) 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 -- 172.21.15.116:0/3515155207 learned_addr learned my addr 172.21.15.116:0/3515155207 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:04.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f581b59e640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 -- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 msgr2=0x7f581c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 -- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c06ab60 msgr2=0x7f581c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f581ad9d640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c06ab60 0x7f581c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c06ab60 0x7f581c150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 -- 172.21.15.116:0/3515155207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040094d0 con 0x7f581c06fa70 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f5813fff640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 0x7f581c1559e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5804000c00 tx=0x7f58040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580403ac60 con 0x7f581c06fa70 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580403adc0 con 0x7f581c06fa70 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581c156930 con 0x7f581c06fa70 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804044630 con 0x7f581c06fa70 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.564+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c156ef0 con 0x7f581c06fa70 2024-02-18T11:48:04.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.565+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c074520 con 0x7f581c06fa70 2024-02-18T11:48:04.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.565+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f580402f050 con 0x7f581c06fa70 2024-02-18T11:48:04.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.566+0000 7f58137fe640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57fc06bc70 0x7f57fc06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.566+0000 7f581ad9d640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57fc06bc70 0x7f57fc06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.566+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f58040acac0 con 0x7f581c06fa70 2024-02-18T11:48:04.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.566+0000 7f581ad9d640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57fc06bc70 0x7f57fc06e130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f580c00a770 tx=0x7f580c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.570+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5804076f60 con 0x7f581c06fa70 2024-02-18T11:48:04.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.743+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f581c0728b0 con 0x7f581c06fa70 2024-02-18T11:48:04.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.745+0000 7f58137fe640 1 -- 172.21.15.116:0/3515155207 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f580407a620 con 0x7f581c06fa70 2024-02-18T11:48:04.738 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:04.738 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:04.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.746+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57fc06bc70 msgr2=0x7f57fc06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57fc06bc70 0x7f57fc06e130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f580c00a770 tx=0x7f580c008040 comp rx=0 tx=0).stop 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 msgr2=0x7f581c1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 0x7f581c1559e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5804000c00 tx=0x7f58040046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 shutdown_connections 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57fc06bc70 0x7f57fc06e130 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f581c160e90 0x7f581c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581c06fa70 0x7f581c1559e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 --2- 172.21.15.116:0/3515155207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f581c06ab60 0x7f581c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 >> 172.21.15.116:0/3515155207 conn(0x7f581c0b23f0 msgr2=0x7f581c0b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 shutdown_connections 2024-02-18T11:48:04.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.747+0000 7f582116b640 1 -- 172.21.15.116:0/3515155207 wait complete. 2024-02-18T11:48:04.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:04.752 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:04.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.863+0000 7f3a53bbc640 1 Processor -- start 2024-02-18T11:48:04.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.863+0000 7f3a53bbc640 1 -- start start 2024-02-18T11:48:04.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a53bbc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a4c0d0080 0x7f3a4c1664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a53bbc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c0703c0 con 0x7f3a4c15cb20 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a53bbc640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c0cd820 con 0x7f3a4c0afc00 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a53bbc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c0affe0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a52159640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a4c0d0080 0x7f3a4c1664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a51157640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3a4c15cb20 0x7f3a4c0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46098/0 (socket says 172.21.15.116:46098) 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.864+0000 7f3a51157640 1 -- 172.21.15.116:0/666757794 learned_addr learned my addr 172.21.15.116:0/666757794 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a52159640 1 -- 172.21.15.116:0/666757794 >> v1:172.21.15.143:6789/0 conn(0x7f3a4c0afc00 legacy=0x7f3a4c06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a52159640 1 -- 172.21.15.116:0/666757794 >> v1:172.21.15.116:6789/0 conn(0x7f3a4c15cb20 legacy=0x7f3a4c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a52159640 1 -- 172.21.15.116:0/666757794 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0ccef0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a52159640 1 --2- 172.21.15.116:0/666757794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c1664d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009d90 tx=0x7f3a4402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31c28814ee110e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a50956640 1 -- 172.21.15.116:0/666757794 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4402cbd0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a50956640 1 -- 172.21.15.116:0/666757794 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4402cd30 con 0x7f3a4c0d0080 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.865+0000 7f3a50956640 1 -- 172.21.15.116:0/666757794 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4402cea0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 -- 172.21.15.116:0/666757794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 msgr2=0x7f3a4c1664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/666757794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c1664d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3a44009d90 tx=0x7f3a4402be20 comp rx=0 tx=0).stop 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 -- 172.21.15.116:0/666757794 shutdown_connections 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/666757794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c1664d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 -- 172.21.15.116:0/666757794 >> 172.21.15.116:0/666757794 conn(0x7f3a4c0b2230 msgr2=0x7f3a4c0b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 -- 172.21.15.116:0/666757794 shutdown_connections 2024-02-18T11:48:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.866+0000 7f3a53bbc640 1 -- 172.21.15.116:0/666757794 wait complete. 2024-02-18T11:48:04.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.867+0000 7f3a53bbc640 1 Processor -- start 2024-02-18T11:48:04.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.867+0000 7f3a53bbc640 1 -- start start 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a53bbc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a4c0afc00 0x7f3a4c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a53bbc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a53bbc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a4c15cb20 0x7f3a4c145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a53bbc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0ca210 con 0x7f3a4c0d0080 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a53bbc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0ca6a0 con 0x7f3a4c15cb20 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a53bbc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0c9600 con 0x7f3a4c0afc00 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c0c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57730/0 (socket says 172.21.15.116:57730) 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 -- 172.21.15.116:0/4163005294 learned_addr learned my addr 172.21.15.116:0/4163005294 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a52159640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a4c15cb20 0x7f3a4c145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51958640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a4c0afc00 0x7f3a4c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 -- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a4c0afc00 msgr2=0x7f3a4c15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a4c0afc00 0x7f3a4c15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 -- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a4c15cb20 msgr2=0x7f3a4c145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a4c15cb20 0x7f3a4c145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a51157640 1 -- 172.21.15.116:0/4163005294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44009a70 con 0x7f3a4c0d0080 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.868+0000 7f3a52159640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a4c15cb20 0x7f3a4c145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.869+0000 7f3a51157640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c0c78d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c480 tx=0x7f3a3c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.869+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c005ce0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.869+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c005e40 con 0x7f3a4c0d0080 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.869+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0072d0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.869+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c0c98e0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.869+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c0c9ea0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.870+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c0ca380 con 0x7f3a4c0d0080 2024-02-18T11:48:04.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.870+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3a3c007430 con 0x7f3a4c0d0080 2024-02-18T11:48:04.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.870+0000 7f3a427fc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3a3006bca0 0x7f3a3006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:04.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.870+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3a3c089ff0 con 0x7f3a4c0d0080 2024-02-18T11:48:04.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.871+0000 7f3a51958640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3a3006bca0 0x7f3a3006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:04.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.871+0000 7f3a51958640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3a3006bca0 0x7f3a3006e160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a48001500 tx=0x7f3a48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:04.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:04.875+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a3c0544a0 con 0x7f3a4c0d0080 2024-02-18T11:48:05.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.046+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a4c074520 con 0x7f3a4c0d0080 2024-02-18T11:48:05.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.046+0000 7f3a427fc640 1 -- 172.21.15.116:0/4163005294 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a3c057b60 con 0x7f3a4c0d0080 2024-02-18T11:48:05.040 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:05.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3a3006bca0 msgr2=0x7f3a3006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3a3006bca0 0x7f3a3006e160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3a48001500 tx=0x7f3a48009040 comp rx=0 tx=0).stop 2024-02-18T11:48:05.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 msgr2=0x7f3a4c0c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c0c78d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c480 tx=0x7f3a3c00c940 comp rx=0 tx=0).stop 2024-02-18T11:48:05.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 shutdown_connections 2024-02-18T11:48:05.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3a3006bca0 0x7f3a3006e160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a4c15cb20 0x7f3a4c145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a4c0d0080 0x7f3a4c0c78d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 --2- 172.21.15.116:0/4163005294 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a4c0afc00 0x7f3a4c15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 >> 172.21.15.116:0/4163005294 conn(0x7f3a4c0b2230 msgr2=0x7f3a4c06e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 shutdown_connections 2024-02-18T11:48:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.048+0000 7f3a53bbc640 1 -- 172.21.15.116:0/4163005294 wait complete. 2024-02-18T11:48:05.053 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:05.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.162+0000 7f457692c640 1 Processor -- start 2024-02-18T11:48:05.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.162+0000 7f457692c640 1 -- start start 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f457692c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f45700d0a80 0x7f45701618c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f457692c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4570099a50 con 0x7f457009ad40 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f457692c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4570099ca0 con 0x7f45700cd610 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f457692c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4570099f80 con 0x7f45700d0a80 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f4574ec9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f457009ad40 0x7f45700cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46102/0 (socket says 172.21.15.116:46102) 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f4574ec9640 1 -- 172.21.15.116:0/1918701711 learned_addr learned my addr 172.21.15.116:0/1918701711 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f456f7fe640 1 --2- 172.21.15.116:0/1918701711 >> v2:172.21.15.143:3300/0 conn(0x7f45700d0a80 0x7f45701618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f456effd640 1 -- 172.21.15.116:0/1918701711 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695540044 0 0) 0x7f4570099a50 con 0x7f457009ad40 2024-02-18T11:48:05.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.163+0000 7f456effd640 1 -- 172.21.15.116:0/1918701711 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4558003520 con 0x7f457009ad40 2024-02-18T11:48:05.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.164+0000 7f456f7fe640 1 -- 172.21.15.116:0/1918701711 >> v1:172.21.15.143:6789/0 conn(0x7f45700cd610 legacy=0x7f45700d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.164+0000 7f456f7fe640 1 -- 172.21.15.116:0/1918701711 >> v1:172.21.15.116:6789/0 conn(0x7f457009ad40 legacy=0x7f45700cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.164+0000 7f456f7fe640 1 -- 172.21.15.116:0/1918701711 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457009a260 con 0x7f45700d0a80 2024-02-18T11:48:05.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.164+0000 7f456f7fe640 1 --2- 172.21.15.116:0/1918701711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700d0a80 0x7f45701618c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45600097a0 tx=0x7f456002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d48bf93d1129a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f456effd640 1 -- 172.21.15.116:0/1918701711 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456002f810 con 0x7f45700d0a80 2024-02-18T11:48:05.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f456effd640 1 -- 172.21.15.116:0/1918701711 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456002f970 con 0x7f45700d0a80 2024-02-18T11:48:05.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f456effd640 1 -- 172.21.15.116:0/1918701711 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456002fce0 con 0x7f45700d0a80 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 -- 172.21.15.116:0/1918701711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700d0a80 msgr2=0x7f45701618c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 --2- 172.21.15.116:0/1918701711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700d0a80 0x7f45701618c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45600097a0 tx=0x7f456002b9f0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 -- 172.21.15.116:0/1918701711 shutdown_connections 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 --2- 172.21.15.116:0/1918701711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700d0a80 0x7f45701618c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 -- 172.21.15.116:0/1918701711 >> 172.21.15.116:0/1918701711 conn(0x7f45700b31f0 msgr2=0x7f45700b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 -- 172.21.15.116:0/1918701711 shutdown_connections 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.165+0000 7f457692c640 1 -- 172.21.15.116:0/1918701711 wait complete. 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.166+0000 7f457692c640 1 Processor -- start 2024-02-18T11:48:05.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.166+0000 7f457692c640 1 -- start start 2024-02-18T11:48:05.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f457692c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f457009ad40 0x7f4570156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f457692c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700cd610 0x7f45700b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f457692c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 0x7f45700b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f457692c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45701f7740 con 0x7f45700d0a80 2024-02-18T11:48:05.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f457692c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45701f7b60 con 0x7f45700cd610 2024-02-18T11:48:05.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f457692c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45701f3a70 con 0x7f457009ad40 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f4574ec9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 0x7f45700b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f4574ec9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 0x7f45700b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57744/0 (socket says 172.21.15.116:57744) 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f4574ec9640 1 -- 172.21.15.116:0/4204433721 learned_addr learned my addr 172.21.15.116:0/4204433721 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.167+0000 7f456f7fe640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700cd610 0x7f45700b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f456ffff640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f457009ad40 0x7f4570156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f4574ec9640 1 -- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f457009ad40 msgr2=0x7f4570156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f4574ec9640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f457009ad40 0x7f4570156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f4574ec9640 1 -- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700cd610 msgr2=0x7f45700b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f4574ec9640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700cd610 0x7f45700b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f4574ec9640 1 -- 172.21.15.116:0/4204433721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4560009400 con 0x7f45700d0a80 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f456f7fe640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700cd610 0x7f45700b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f456ffff640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f457009ad40 0x7f4570156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f4574ec9640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 0x7f45700b6430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f456800cb40 tx=0x7f456800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800aca0 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800ae00 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568012620 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45701f6da0 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.168+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45701f7360 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.169+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4570067840 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.169+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4568007070 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.170+0000 7f456cff9640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f455c06bc70 0x7f455c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.170+0000 7f456ffff640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f455c06bc70 0x7f455c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.170+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f456808ab00 con 0x7f45700d0a80 2024-02-18T11:48:05.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.170+0000 7f456ffff640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f455c06bc70 0x7f455c06e130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4564001680 tx=0x7f4564009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.174+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4568055030 con 0x7f45700d0a80 2024-02-18T11:48:05.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.367+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4570067420 con 0x7f45700d0a80 2024-02-18T11:48:05.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.367+0000 7f456cff9640 1 -- 172.21.15.116:0/4204433721 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45680586f0 con 0x7f45700d0a80 2024-02-18T11:48:05.361 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f455c06bc70 msgr2=0x7f455c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f455c06bc70 0x7f455c06e130 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4564001680 tx=0x7f4564009040 comp rx=0 tx=0).stop 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 msgr2=0x7f45700b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 0x7f45700b6430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f456800cb40 tx=0x7f456800a060 comp rx=0 tx=0).stop 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 shutdown_connections 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f455c06bc70 0x7f455c06e130 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45700d0a80 0x7f45700b6430 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45700cd610 0x7f45700b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 --2- 172.21.15.116:0/4204433721 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f457009ad40 0x7f4570156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 >> 172.21.15.116:0/4204433721 conn(0x7f45700b31f0 msgr2=0x7f45700ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 shutdown_connections 2024-02-18T11:48:05.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.369+0000 7f457692c640 1 -- 172.21.15.116:0/4204433721 wait complete. 2024-02-18T11:48:05.373 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:05.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.483+0000 7fc397e15640 1 Processor -- start 2024-02-18T11:48:05.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc397e15640 1 -- start start 2024-02-18T11:48:05.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc397e15640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc39015dbf0 0x7fc3901617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc397e15640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900cee50 0x7fc3900d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc397e15640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3900d1ee0 con 0x7fc3900b9370 2024-02-18T11:48:05.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc397e15640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc390161dc0 con 0x7fc39015dbf0 2024-02-18T11:48:05.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc397e15640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3901620a0 con 0x7fc3900cee50 2024-02-18T11:48:05.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.484+0000 7fc395bb1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc39015dbf0 0x7fc3901617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc39015dbf0 0x7fc3901617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57756/0 (socket says 172.21.15.116:57756) 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc3953b0640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc3900b9370 0x7fc3900ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46108/0 (socket says 172.21.15.116:46108) 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 -- 172.21.15.116:0/3676567340 learned_addr learned my addr 172.21.15.116:0/3676567340 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc3963b2640 1 --2- 172.21.15.116:0/3676567340 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900cee50 0x7fc3900d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 -- 172.21.15.116:0/3676567340 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900cee50 msgr2=0x7fc3900d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 --2- 172.21.15.116:0/3676567340 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900cee50 0x7fc3900d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 -- 172.21.15.116:0/3676567340 >> v1:172.21.15.116:6789/0 conn(0x7fc3900b9370 legacy=0x7fc3900ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 -- 172.21.15.116:0/3676567340 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390162380 con 0x7fc39015dbf0 2024-02-18T11:48:05.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc394baf640 1 -- 172.21.15.116:0/3676567340 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197887522 0 0) 0x7fc3900d1ee0 con 0x7fc3900b9370 2024-02-18T11:48:05.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.485+0000 7fc395bb1640 1 --2- 172.21.15.116:0/3676567340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39015dbf0 0x7fc3901617e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009e50 tx=0x7fc38c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17580c12cdd6221 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc394baf640 1 -- 172.21.15.116:0/3676567340 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c02cb00 con 0x7fc39015dbf0 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc394baf640 1 -- 172.21.15.116:0/3676567340 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c02cc60 con 0x7fc39015dbf0 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc394baf640 1 -- 172.21.15.116:0/3676567340 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c0343f0 con 0x7fc39015dbf0 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 -- 172.21.15.116:0/3676567340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39015dbf0 msgr2=0x7fc3901617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 --2- 172.21.15.116:0/3676567340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39015dbf0 0x7fc3901617e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009e50 tx=0x7fc38c02bf60 comp rx=0 tx=0).stop 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 -- 172.21.15.116:0/3676567340 shutdown_connections 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 --2- 172.21.15.116:0/3676567340 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900cee50 0x7fc3900d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 --2- 172.21.15.116:0/3676567340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39015dbf0 0x7fc3901617e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 -- 172.21.15.116:0/3676567340 >> 172.21.15.116:0/3676567340 conn(0x7fc3900b5480 msgr2=0x7fc3900b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 -- 172.21.15.116:0/3676567340 shutdown_connections 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.486+0000 7fc397e15640 1 -- 172.21.15.116:0/3676567340 wait complete. 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.487+0000 7fc397e15640 1 Processor -- start 2024-02-18T11:48:05.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.487+0000 7fc397e15640 1 -- start start 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc397e15640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900b9370 0x7fc3901f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc397e15640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 0x7fc3901f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc397e15640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39015dbf0 0x7fc39015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc397e15640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39015d340 con 0x7fc3900cee50 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc397e15640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39015b760 con 0x7fc39015dbf0 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc397e15640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39015ba20 con 0x7fc3900b9370 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc3953b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 0x7fc3901f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc3953b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 0x7fc3901f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57762/0 (socket says 172.21.15.116:57762) 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc3953b0640 1 -- 172.21.15.116:0/3199782819 learned_addr learned my addr 172.21.15.116:0/3199782819 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc3963b2640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39015dbf0 0x7fc39015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.488+0000 7fc395bb1640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900b9370 0x7fc3901f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3953b0640 1 -- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900b9370 msgr2=0x7fc3901f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3953b0640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900b9370 0x7fc3901f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3953b0640 1 -- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39015dbf0 msgr2=0x7fc39015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3953b0640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39015dbf0 0x7fc39015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3953b0640 1 -- 172.21.15.116:0/3199782819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c009b30 con 0x7fc3900cee50 2024-02-18T11:48:05.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3953b0640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 0x7fc3901f6230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc38000c430 tx=0x7fc38000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800074e0 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380007660 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39015e150 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380004a20 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.489+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39015f7e0 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.490+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc390067840 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.490+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc380004b80 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.491+0000 7fc3867fc640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc37406bc70 0x7fc37406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.491+0000 7fc395bb1640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc37406bc70 0x7fc37406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.491+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc38008a790 con 0x7fc3900cee50 2024-02-18T11:48:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.491+0000 7fc395bb1640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc37406bc70 0x7fc37406e130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc38c035000 tx=0x7fc38c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.495+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc380054cc0 con 0x7fc3900cee50 2024-02-18T11:48:05.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.668+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc390074520 con 0x7fc3900cee50 2024-02-18T11:48:05.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.668+0000 7fc3867fc640 1 -- 172.21.15.116:0/3199782819 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc380058380 con 0x7fc3900cee50 2024-02-18T11:48:05.662 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc37406bc70 msgr2=0x7fc37406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc37406bc70 0x7fc37406e130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc38c035000 tx=0x7fc38c030000 comp rx=0 tx=0).stop 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 msgr2=0x7fc3901f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 0x7fc3901f6230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc38000c430 tx=0x7fc38000c8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 shutdown_connections 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc37406bc70 0x7fc37406e130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39015dbf0 0x7fc39015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3900cee50 0x7fc3901f6230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 --2- 172.21.15.116:0/3199782819 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc3900b9370 0x7fc3901f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 >> 172.21.15.116:0/3199782819 conn(0x7fc3900b5480 msgr2=0x7fc3900cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 shutdown_connections 2024-02-18T11:48:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.670+0000 7fc397e15640 1 -- 172.21.15.116:0/3199782819 wait complete. 2024-02-18T11:48:05.675 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:05.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.785+0000 7f2bcde2e640 1 Processor -- start 2024-02-18T11:48:05.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.785+0000 7f2bcde2e640 1 -- start start 2024-02-18T11:48:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bcde2e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bcde2e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2bc80b9370 0x7f2bc80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bcde2e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc80d1ee0 con 0x7f2bc80cec80 2024-02-18T11:48:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bcde2e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc80cf110 con 0x7f2bc80b9370 2024-02-18T11:48:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bcde2e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8161dc0 con 0x7f2bc815dbf0 2024-02-18T11:48:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bc6ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2bc80b9370 0x7f2bc80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bc77fe640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bc77fe640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc81617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:56968/0 (socket says 172.21.15.116:56968) 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bc77fe640 1 -- 172.21.15.116:0/4131272132 learned_addr learned my addr 172.21.15.116:0/4131272132 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.786+0000 7f2bc6ffd640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2bc80b9370 0x7f2bc80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:54428/0 (socket says 172.21.15.116:54428) 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 msgr2=0x7f2bc81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc6ffd640 1 --2- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/4131272132 >> v1:172.21.15.143:6789/0 conn(0x7f2bc80cec80 legacy=0x7f2bc806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/4131272132 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc81620a0 con 0x7f2bc80b9370 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc67fc640 1 -- 172.21.15.116:0/4131272132 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782198545 0 0) 0x7f2bc80d1ee0 con 0x7f2bc80cec80 2024-02-18T11:48:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc77fe640 1 --2- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.787+0000 7f2bc6ffd640 1 --2- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc80ce740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0097a0 tx=0x7f2bbc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b550ba8c38ce994a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bc67fc640 1 -- 172.21.15.116:0/4131272132 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc031830 con 0x7f2bc80b9370 2024-02-18T11:48:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bc67fc640 1 -- 172.21.15.116:0/4131272132 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc031990 con 0x7f2bc80b9370 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bc67fc640 1 -- 172.21.15.116:0/4131272132 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc031d20 con 0x7f2bc80b9370 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 -- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 msgr2=0x7f2bc80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc80ce740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0097a0 tx=0x7f2bbc02b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 -- 172.21.15.116:0/4131272132 shutdown_connections 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc80ce740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/4131272132 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 -- 172.21.15.116:0/4131272132 >> 172.21.15.116:0/4131272132 conn(0x7f2bc80b5480 msgr2=0x7f2bc80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 -- 172.21.15.116:0/4131272132 shutdown_connections 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.788+0000 7f2bcde2e640 1 -- 172.21.15.116:0/4131272132 wait complete. 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.789+0000 7f2bcde2e640 1 Processor -- start 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 -- start start 2024-02-18T11:48:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc81f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 0x7f2bc8154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc815a9f0 con 0x7f2bc80cec80 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8159e70 con 0x7f2bc80b9370 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bcde2e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc815a150 con 0x7f2bc815dbf0 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bc6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 0x7f2bc8154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bc6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 0x7f2bc8154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57766/0 (socket says 172.21.15.116:57766) 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.790+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/136501145 learned_addr learned my addr 172.21.15.116:0/136501145 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc77fe640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc81f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc7fff640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 msgr2=0x7f2bc8158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc6ffd640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc8158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 msgr2=0x7f2bc81f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc6ffd640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc81f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc6ffd640 1 -- 172.21.15.116:0/136501145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80094d0 con 0x7f2bc80cec80 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc77fe640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc81f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bc6ffd640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 0x7f2bc8154a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0098c0 tx=0x7f2bbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc03a6f0 con 0x7f2bc80cec80 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc03a870 con 0x7f2bc80cec80 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc009400 con 0x7f2bc80cec80 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc03ac20 con 0x7f2bc80cec80 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.791+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc815e150 con 0x7f2bc80cec80 2024-02-18T11:48:05.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.792+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8067840 con 0x7f2bc80cec80 2024-02-18T11:48:05.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.792+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f2bbc030380 con 0x7f2bc80cec80 2024-02-18T11:48:05.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.793+0000 7f2baffff640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2bb406bc70 0x7f2bb406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:05.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.793+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2bbc0ac4b0 con 0x7f2bc80cec80 2024-02-18T11:48:05.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.793+0000 7f2bc77fe640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2bb406bc70 0x7f2bb406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:05.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.793+0000 7f2bc77fe640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2bb406bc70 0x7f2bb406e130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2bc815bc80 tx=0x7f2bb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:05.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.797+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bbc076a80 con 0x7f2bc80cec80 2024-02-18T11:48:05.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.966+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2bc8074520 con 0x7f2bc80cec80 2024-02-18T11:48:05.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.966+0000 7f2baffff640 1 -- 172.21.15.116:0/136501145 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2bbc07a140 con 0x7f2bc80cec80 2024-02-18T11:48:05.960 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2bb406bc70 msgr2=0x7f2bb406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2bb406bc70 0x7f2bb406e130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2bc815bc80 tx=0x7f2bb8008040 comp rx=0 tx=0).stop 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 msgr2=0x7f2bc8154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 0x7f2bc8154a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0098c0 tx=0x7f2bbc0046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 shutdown_connections 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2bb406bc70 0x7f2bb406e130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2bc815dbf0 0x7f2bc8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bc80cec80 0x7f2bc8154a70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 --2- 172.21.15.116:0/136501145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bc80b9370 0x7f2bc81f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:05.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 >> 172.21.15.116:0/136501145 conn(0x7f2bc80b5480 msgr2=0x7f2bc80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:05.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 shutdown_connections 2024-02-18T11:48:05.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:05.968+0000 7f2bcde2e640 1 -- 172.21.15.116:0/136501145 wait complete. 2024-02-18T11:48:05.973 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.080+0000 7fe9c26df640 1 Processor -- start 2024-02-18T11:48:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.080+0000 7fe9c26df640 1 -- start start 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9c26df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9bc15dc60 0x7fe9bc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9c26df640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9bc161850 0x7fe9bc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9c26df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0ce780 con 0x7fe9bc0d0970 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9c26df640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc0d3730 con 0x7fe9bc161850 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9c26df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc0cf250 con 0x7fe9bc15dc60 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9bb7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9bc161850 0x7fe9bc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9bb7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9bc161850 0x7fe9bc0ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57774/0 (socket says 172.21.15.116:57774) 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.081+0000 7fe9bb7fe640 1 -- 172.21.15.116:0/821451222 learned_addr learned my addr 172.21.15.116:0/821451222 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bbfff640 1 --2- 172.21.15.116:0/821451222 >> v2:172.21.15.143:3300/0 conn(0x7fe9bc15dc60 0x7fe9bc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bb7fe640 1 -- 172.21.15.116:0/821451222 >> v2:172.21.15.143:3300/0 conn(0x7fe9bc15dc60 msgr2=0x7fe9bc0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bb7fe640 1 --2- 172.21.15.116:0/821451222 >> v2:172.21.15.143:3300/0 conn(0x7fe9bc15dc60 0x7fe9bc0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bb7fe640 1 -- 172.21.15.116:0/821451222 >> v1:172.21.15.143:6789/0 conn(0x7fe9bc0d0970 legacy=0x7fe9bc0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bb7fe640 1 -- 172.21.15.116:0/821451222 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0ce9b0 con 0x7fe9bc161850 2024-02-18T11:48:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bbfff640 1 --2- 172.21.15.116:0/821451222 >> v2:172.21.15.143:3300/0 conn(0x7fe9bc15dc60 0x7fe9bc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.082+0000 7fe9bb7fe640 1 --2- 172.21.15.116:0/821451222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc161850 0x7fe9bc0ce240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc08c680 tx=0x7fe9a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abeeeef6f75fcd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9baffd640 1 -- 172.21.15.116:0/821451222 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4031870 con 0x7fe9bc161850 2024-02-18T11:48:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9baffd640 1 -- 172.21.15.116:0/821451222 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a40319d0 con 0x7fe9bc161850 2024-02-18T11:48:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9baffd640 1 -- 172.21.15.116:0/821451222 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4031d60 con 0x7fe9bc161850 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 -- 172.21.15.116:0/821451222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc161850 msgr2=0x7fe9bc0ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 --2- 172.21.15.116:0/821451222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc161850 0x7fe9bc0ce240 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc08c680 tx=0x7fe9a402b910 comp rx=0 tx=0).stop 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 -- 172.21.15.116:0/821451222 shutdown_connections 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 --2- 172.21.15.116:0/821451222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc161850 0x7fe9bc0ce240 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 --2- 172.21.15.116:0/821451222 >> v2:172.21.15.143:3300/0 conn(0x7fe9bc15dc60 0x7fe9bc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 -- 172.21.15.116:0/821451222 >> 172.21.15.116:0/821451222 conn(0x7fe9bc0b5910 msgr2=0x7fe9bc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 -- 172.21.15.116:0/821451222 shutdown_connections 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.083+0000 7fe9c26df640 1 -- 172.21.15.116:0/821451222 wait complete. 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.084+0000 7fe9c26df640 1 Processor -- start 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 -- start start 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 0x7fe9bc160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9bc15dc60 0x7fe9bc0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9bc161850 0x7fe9bc156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1569a0 con 0x7fe9bc0d0970 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1611b0 con 0x7fe9bc15dc60 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c26df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1541d0 con 0x7fe9bc161850 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 0x7fe9bc160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9bbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 0x7fe9bc160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57790/0 (socket says 172.21.15.116:57790) 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9bbfff640 1 -- 172.21.15.116:0/3673830306 learned_addr learned my addr 172.21.15.116:0/3673830306 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9bb7fe640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9bc15dc60 0x7fe9bc0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.085+0000 7fe9c0c7c640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9bc161850 0x7fe9bc156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9bbfff640 1 -- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9bc161850 msgr2=0x7fe9bc156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9bbfff640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9bc161850 0x7fe9bc156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9bbfff640 1 -- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9bc15dc60 msgr2=0x7fe9bc0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9bbfff640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9bc15dc60 0x7fe9bc0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9bbfff640 1 -- 172.21.15.116:0/3673830306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a40094d0 con 0x7fe9bc0d0970 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9bbfff640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 0x7fe9bc160c70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00e840 tx=0x7fe9ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac004030 con 0x7fe9bc0d0970 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac004190 con 0x7fe9bc0d0970 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc155d20 con 0x7fe9bc0d0970 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac004520 con 0x7fe9bc0d0970 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.086+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc1562e0 con 0x7fe9bc0d0970 2024-02-18T11:48:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.087+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc067840 con 0x7fe9bc0d0970 2024-02-18T11:48:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.087+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe9ac00c6d0 con 0x7fe9bc0d0970 2024-02-18T11:48:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.088+0000 7fe9b8ff9640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe99c06bc70 0x7fe99c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.088+0000 7fe9bb7fe640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe99c06bc70 0x7fe99c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.088+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe9ac08a710 con 0x7fe9bc0d0970 2024-02-18T11:48:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.088+0000 7fe9bb7fe640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe99c06bc70 0x7fe99c06e130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4036000 tx=0x7fe9a403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.092+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9ac054c40 con 0x7fe9bc0d0970 2024-02-18T11:48:06.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.299+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9bc074520 con 0x7fe9bc0d0970 2024-02-18T11:48:06.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.299+0000 7fe9b8ff9640 1 -- 172.21.15.116:0/3673830306 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fe9ac058300 con 0x7fe9bc0d0970 2024-02-18T11:48:06.293 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:06.293 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:06.293 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:06.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe99c06bc70 msgr2=0x7fe99c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.295 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe99c06bc70 0x7fe99c06e130 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4036000 tx=0x7fe9a403c040 comp rx=0 tx=0).stop 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 msgr2=0x7fe9bc160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 0x7fe9bc160c70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00e840 tx=0x7fe9ac00ed00 comp rx=0 tx=0).stop 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 shutdown_connections 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe99c06bc70 0x7fe99c06e130 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9bc161850 0x7fe9bc156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9bc15dc60 0x7fe9bc0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 --2- 172.21.15.116:0/3673830306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9bc0d0970 0x7fe9bc160c70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.301+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 >> 172.21.15.116:0/3673830306 conn(0x7fe9bc0b5910 msgr2=0x7fe9bc15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.302+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 shutdown_connections 2024-02-18T11:48:06.296 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.302+0000 7fe9c26df640 1 -- 172.21.15.116:0/3673830306 wait complete. 2024-02-18T11:48:06.308 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.415+0000 7fd7460c3640 1 Processor -- start 2024-02-18T11:48:06.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.415+0000 7fd7460c3640 1 -- start start 2024-02-18T11:48:06.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd7460c3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 0x7fd7400d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd7460c3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400d0a80 con 0x7fd74015efb0 2024-02-18T11:48:06.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd7460c3640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400cfe70 con 0x7fd740099a50 2024-02-18T11:48:06.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd7460c3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74015f390 con 0x7fd7400cf900 2024-02-18T11:48:06.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73f7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd74015efb0 0x7fd74006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46110/0 (socket says 172.21.15.116:46110) 2024-02-18T11:48:06.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73f7fe640 1 -- 172.21.15.116:0/3333985154 learned_addr learned my addr 172.21.15.116:0/3333985154 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522578895 0 0) 0x7fd7400d0a80 con 0x7fd74015efb0 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd728003520 con 0x7fd74015efb0 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73effd640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 0x7fd7400d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1983244892 0 0) 0x7fd728003520 con 0x7fd74015efb0 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 msgr2=0x7fd7400d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 0x7fd7400d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 >> v1:172.21.15.143:6789/0 conn(0x7fd740099a50 legacy=0x7fd7400cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400d0150 con 0x7fd74015efb0 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fd730001670 con 0x7fd74015efb0 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 >> v1:172.21.15.116:6789/0 conn(0x7fd74015efb0 legacy=0x7fd74006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.416+0000 7fd73e7fc640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7280039a0 0x7fd728003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73e7fc640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 0x7fd7280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73e7fc640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd728004d00 0x7fd728005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74015f390 con 0x7fd7280039a0 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72800bab0 con 0x7fd728004d00 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72800bc70 con 0x7fd7400cf900 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7280039a0 0x7fd728003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73f7fe640 1 -- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 msgr2=0x7fd7280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7400cf900 0x7fd7280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73f7fe640 1 -- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd728004d00 msgr2=0x7fd728005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd728004d00 0x7fd728005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.417+0000 7fd73f7fe640 1 -- 172.21.15.116:0/3333985154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728003520 con 0x7fd7280039a0 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7280039a0 0x7fd728003dc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c190 tx=0x7fd7280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7972632ca3d276d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd730005ad0 con 0x7fd7280039a0 2024-02-18T11:48:06.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730005c30 con 0x7fd7280039a0 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd73e7fc640 1 -- 172.21.15.116:0/3333985154 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73002c7e0 con 0x7fd7280039a0 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd7460c3640 1 -- 172.21.15.116:0/3333985154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7280039a0 msgr2=0x7fd728003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd7460c3640 1 --2- 172.21.15.116:0/3333985154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7280039a0 0x7fd728003dc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c190 tx=0x7fd7280057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd73ffff640 1 -- 172.21.15.116:0/3333985154 reap_dead start 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd7460c3640 1 -- 172.21.15.116:0/3333985154 shutdown_connections 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd7460c3640 1 -- 172.21.15.116:0/3333985154 >> 172.21.15.116:0/3333985154 conn(0x7fd7400d1790 msgr2=0x7fd7400d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd7460c3640 1 -- 172.21.15.116:0/3333985154 shutdown_connections 2024-02-18T11:48:06.415 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.418+0000 7fd7460c3640 1 -- 172.21.15.116:0/3333985154 wait complete. 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.419+0000 7fd7460c3640 1 Processor -- start 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 -- start start 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7280039a0 0x7fd740155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 0x7fd7400b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74015dbd0 0x7fd74015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400b7170 con 0x7fd7400b67b0 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c6f50 con 0x7fd74015dbd0 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd7460c3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74015e590 con 0x7fd7280039a0 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd73effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 0x7fd7400b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.418 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd73effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 0x7fd7400b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57800/0 (socket says 172.21.15.116:57800) 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.420+0000 7fd73effd640 1 -- 172.21.15.116:0/1939506225 learned_addr learned my addr 172.21.15.116:0/1939506225 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7280039a0 0x7fd740155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73ffff640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74015dbd0 0x7fd74015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73effd640 1 -- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7280039a0 msgr2=0x7fd740155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73effd640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7280039a0 0x7fd740155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73effd640 1 -- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74015dbd0 msgr2=0x7fd74015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73effd640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74015dbd0 0x7fd74015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73effd640 1 -- 172.21.15.116:0/1939506225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400c6380 con 0x7fd7400b67b0 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd73effd640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 0x7fd7400b6c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e820 tx=0x7fd73400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400c8e0 con 0x7fd7400b67b0 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73400ca40 con 0x7fd7400b67b0 2024-02-18T11:48:06.419 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74015a9e0 con 0x7fd7400b67b0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400cdd0 con 0x7fd7400b67b0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.421+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7400c6900 con 0x7fd7400b67b0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.422+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd740067840 con 0x7fd7400b67b0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.422+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd734013470 con 0x7fd7400b67b0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.423+0000 7fd727fff640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd72c06bca0 0x7fd72c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.423+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd72c06bca0 0x7fd72c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.423+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd73408aa50 con 0x7fd7400b67b0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.423+0000 7fd73f7fe640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd72c06bca0 0x7fd72c06e160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd730034000 tx=0x7fd730009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.420 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.427+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd734054ef0 con 0x7fd7400b67b0 2024-02-18T11:48:06.595 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.601+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd740074520 con 0x7fd7400b67b0 2024-02-18T11:48:06.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.602+0000 7fd727fff640 1 -- 172.21.15.116:0/1939506225 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fd7340585b0 con 0x7fd7400b67b0 2024-02-18T11:48:06.596 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:06.596 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":24,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:47:38.907502+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd72c06bca0 msgr2=0x7fd72c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd72c06bca0 0x7fd72c06e160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd730034000 tx=0x7fd730009030 comp rx=0 tx=0).stop 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 msgr2=0x7fd7400b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 0x7fd7400b6c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd73400e820 tx=0x7fd73400ece0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 shutdown_connections 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd72c06bca0 0x7fd72c06e160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd74015dbd0 0x7fd74015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7400b67b0 0x7fd7400b6c30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 --2- 172.21.15.116:0/1939506225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7280039a0 0x7fd740155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.604+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 >> 172.21.15.116:0/1939506225 conn(0x7fd7400d1790 msgr2=0x7fd740070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:06.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.605+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 shutdown_connections 2024-02-18T11:48:06.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.605+0000 7fd7460c3640 1 -- 172.21.15.116:0/1939506225 wait complete. 2024-02-18T11:48:06.609 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.717+0000 7fdfc7bbc640 1 Processor -- start 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.717+0000 7fdfc7bbc640 1 -- start start 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc7bbc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdfc006da20 0x7fdfc015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc7bbc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc00b3e10 con 0x7fdfc00b37d0 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc7bbc640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc01615c0 con 0x7fdfc0158260 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc7bbc640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc00b5360 con 0x7fdfc006da20 2024-02-18T11:48:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc5157640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdfc00b37d0 0x7fdfc006ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:46122/0 (socket says 172.21.15.116:46122) 2024-02-18T11:48:06.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc5157640 1 -- 172.21.15.116:0/3807729187 learned_addr learned my addr 172.21.15.116:0/3807729187 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:06.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc6159640 1 --2- 172.21.15.116:0/3807729187 >> v2:172.21.15.143:3300/0 conn(0x7fdfc006da20 0x7fdfc015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372111652 0 0) 0x7fdfc00b3e10 con 0x7fdfc00b37d0 2024-02-18T11:48:06.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfac003520 con 0x7fdfc00b37d0 2024-02-18T11:48:06.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622918251 0 0) 0x7fdfc01615c0 con 0x7fdfc0158260 2024-02-18T11:48:06.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.718+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc00b3e10 con 0x7fdfc0158260 2024-02-18T11:48:06.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.719+0000 7fdfc6159640 1 -- 172.21.15.116:0/3807729187 >> v1:172.21.15.143:6789/0 conn(0x7fdfc0158260 legacy=0x7fdfc00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.719+0000 7fdfc6159640 1 -- 172.21.15.116:0/3807729187 >> v1:172.21.15.116:6789/0 conn(0x7fdfc00b37d0 legacy=0x7fdfc006ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.719+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671120363 0 0) 0x7fdfac003520 con 0x7fdfc00b37d0 2024-02-18T11:48:06.715 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.719+0000 7fdfc6159640 1 -- 172.21.15.116:0/3807729187 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc01618a0 con 0x7fdfc006da20 2024-02-18T11:48:06.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.719+0000 7fdfc6159640 1 --2- 172.21.15.116:0/3807729187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc015df80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009d90 tx=0x7fdfb802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41d5ed37806592b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0001670 con 0x7fdfc006da20 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb802c940 con 0x7fdfc006da20 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc4956640 1 -- 172.21.15.116:0/3807729187 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb802ccd0 con 0x7fdfc006da20 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/3807729187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 msgr2=0x7fdfc015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/3807729187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc015df80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009d90 tx=0x7fdfb802be10 comp rx=0 tx=0).stop 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/3807729187 shutdown_connections 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/3807729187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc015df80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/3807729187 >> 172.21.15.116:0/3807729187 conn(0x7fdfc00b23d0 msgr2=0x7fdfc00cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/3807729187 shutdown_connections 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.720+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/3807729187 wait complete. 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.721+0000 7fdfc7bbc640 1 Processor -- start 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.721+0000 7fdfc7bbc640 1 -- start start 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc7bbc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc00cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc7bbc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdfc00b37d0 0x7fdfc00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc7bbc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 0x7fdfc0153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc7bbc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0153190 con 0x7fdfc0158260 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc7bbc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc01588a0 con 0x7fdfc006da20 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc7bbc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc0153f90 con 0x7fdfc00b37d0 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 0x7fdfc0153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 0x7fdfc0153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:57802/0 (socket says 172.21.15.116:57802) 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 -- 172.21.15.116:0/326730280 learned_addr learned my addr 172.21.15.116:0/326730280 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc5958640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc00cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc5157640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdfc00b37d0 0x7fdfc00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 -- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdfc00b37d0 msgr2=0x7fdfc00cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdfc00b37d0 0x7fdfc00cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 -- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 msgr2=0x7fdfc00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc00cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.722+0000 7fdfc6159640 1 -- 172.21.15.116:0/326730280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8009a70 con 0x7fdfc0158260 2024-02-18T11:48:06.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.723+0000 7fdfc6159640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 0x7fdfc0153380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009d90 tx=0x7fdfb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.723+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8009450 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.723+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb8034c50 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.723+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc0154270 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.723+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8003990 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.723+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc0154830 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.724+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc0074520 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.724+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fdfb8040450 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.724+0000 7fdfb67fc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdfa406bc70 0x7fdfa406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.725+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdfb80ad4d0 con 0x7fdfc0158260 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.725+0000 7fdfc5958640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdfa406bc70 0x7fdfa406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:06.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.725+0000 7fdfc5958640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdfa406bc70 0x7fdfa406e130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0011e0 tx=0x7fdfbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:06.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.729+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfb8077b40 con 0x7fdfc0158260 2024-02-18T11:48:06.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.935+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfc00cd710 con 0x7fdfc0158260 2024-02-18T11:48:06.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.935+0000 7fdfb67fc640 1 -- 172.21.15.116:0/326730280 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fdfb807b200 con 0x7fdfc0158260 2024-02-18T11:48:06.929 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:06.929 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:06.930 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdfa406bc70 msgr2=0x7fdfa406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdfa406bc70 0x7fdfa406e130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0011e0 tx=0x7fdfbc009040 comp rx=0 tx=0).stop 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 msgr2=0x7fdfc0153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 0x7fdfc0153380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009d90 tx=0x7fdfb8004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 shutdown_connections 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdfa406bc70 0x7fdfa406e130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdfc0158260 0x7fdfc0153380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdfc00b37d0 0x7fdfc00cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 --2- 172.21.15.116:0/326730280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfc006da20 0x7fdfc00cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:06.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 >> 172.21.15.116:0/326730280 conn(0x7fdfc00b23d0 msgr2=0x7fdfc00b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:06.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 shutdown_connections 2024-02-18T11:48:06.932 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:06.937+0000 7fdfc7bbc640 1 -- 172.21.15.116:0/326730280 wait complete. 2024-02-18T11:48:07.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-18T11:47:33.859710+0000', 'modified': '2024-02-18T11:47:55.942466+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/1897601775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 1897601775}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 1897601775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-18T11:48:07.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-18T11:48:07.345 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1de52c640 1 Processor -- start 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1de52c640 1 -- start start 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1de52c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1d815f8c0 0x7ff1d809a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1de52c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80d06c0 con 0x7ff1d815b560 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1de52c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80d03f0 con 0x7ff1d815fca0 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1de52c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d8099bf0 con 0x7ff1d815f8c0 2024-02-18T11:48:07.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.454+0000 7ff1d7fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1d815f8c0 0x7ff1d809a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.455+0000 7ff1dcac9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff1d815b560 0x7ff1d815b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59718/0 (socket says 172.21.15.116:59718) 2024-02-18T11:48:07.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.455+0000 7ff1dcac9640 1 -- 172.21.15.116:0/16601133 learned_addr learned my addr 172.21.15.116:0/16601133 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:07.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.455+0000 7ff1d7fff640 1 -- 172.21.15.116:0/16601133 >> v1:172.21.15.143:6789/0 conn(0x7ff1d815fca0 legacy=0x7ff1d80cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.455+0000 7ff1d7fff640 1 -- 172.21.15.116:0/16601133 >> v1:172.21.15.116:6789/0 conn(0x7ff1d815b560 legacy=0x7ff1d815b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.455+0000 7ff1d7fff640 1 -- 172.21.15.116:0/16601133 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d8099ed0 con 0x7ff1d815f8c0 2024-02-18T11:48:07.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.455+0000 7ff1d6ffd640 1 -- 172.21.15.116:0/16601133 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286343441 0 0) 0x7ff1d0001490 con 0x7ff1d815b560 2024-02-18T11:48:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/16601133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d809a9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00097a0 tx=0x7ff1c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214597c03ac95a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1d6ffd640 1 -- 172.21.15.116:0/16601133 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0031830 con 0x7ff1d815f8c0 2024-02-18T11:48:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1d6ffd640 1 -- 172.21.15.116:0/16601133 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c0031990 con 0x7ff1d815f8c0 2024-02-18T11:48:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1d6ffd640 1 -- 172.21.15.116:0/16601133 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0031d20 con 0x7ff1d815f8c0 2024-02-18T11:48:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 -- 172.21.15.116:0/16601133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 msgr2=0x7ff1d809a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 --2- 172.21.15.116:0/16601133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d809a9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00097a0 tx=0x7ff1c002b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 -- 172.21.15.116:0/16601133 shutdown_connections 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 --2- 172.21.15.116:0/16601133 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d809a9b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 -- 172.21.15.116:0/16601133 >> 172.21.15.116:0/16601133 conn(0x7ff1d80b31f0 msgr2=0x7ff1d80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 -- 172.21.15.116:0/16601133 shutdown_connections 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.456+0000 7ff1de52c640 1 -- 172.21.15.116:0/16601133 wait complete. 2024-02-18T11:48:07.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.457+0000 7ff1de52c640 1 Processor -- start 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 -- start start 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff1d815b560 0x7ff1d815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1d815fca0 0x7ff1d8156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d806c770 con 0x7ff1d815f8c0 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815d280 con 0x7ff1d815fca0 2024-02-18T11:48:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1de52c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d815ec00 con 0x7ff1d815b560 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1d77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1d77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d815e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39460/0 (socket says 172.21.15.116:39460) 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.458+0000 7ff1d77fe640 1 -- 172.21.15.116:0/3637112053 learned_addr learned my addr 172.21.15.116:0/3637112053 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1d815fca0 0x7ff1d8156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff1d815b560 0x7ff1d815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d77fe640 1 -- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff1d815b560 msgr2=0x7ff1d815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff1d815b560 0x7ff1d815cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d77fe640 1 -- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1d815fca0 msgr2=0x7ff1d8156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1d815fca0 0x7ff1d8156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d77fe640 1 -- 172.21.15.116:0/3637112053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c0009400 con 0x7ff1d815f8c0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d815e6c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b3f0 tx=0x7ff1c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d4ff9640 1 -- 172.21.15.116:0/3637112053 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8011830 con 0x7ff1d815f8c0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d4ff9640 1 -- 172.21.15.116:0/3637112053 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8011990 con 0x7ff1d815f8c0 2024-02-18T11:48:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d815eea0 con 0x7ff1d815f8c0 2024-02-18T11:48:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1d4ff9640 1 -- 172.21.15.116:0/3637112053 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8011d40 con 0x7ff1d815f8c0 2024-02-18T11:48:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.459+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d806c120 con 0x7ff1d815f8c0 2024-02-18T11:48:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.460+0000 7ff1d4ff9640 1 -- 172.21.15.116:0/3637112053 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff1c801a400 con 0x7ff1d815f8c0 2024-02-18T11:48:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.461+0000 7ff1d4ff9640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff1b806bc70 0x7ff1b806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.461+0000 7ff1d4ff9640 1 -- 172.21.15.116:0/3637112053 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff1c808a870 con 0x7ff1d815f8c0 2024-02-18T11:48:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.461+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff1b806bc70 0x7ff1b806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.461+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff1b806bc70 0x7ff1b806e130 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0036000 tx=0x7ff1c0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 Processor -- start 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 -- start start 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x560676204480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 0x560676201310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x560676201850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x560676194dc0 con 0x5606761d40c0 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560676147f20 con 0x5606762040a0 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.462+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x560676145450 con 0x5606761d7c20 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 0x560676201310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 0x560676201310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39464/0 (socket says 172.21.15.116:39464) 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 -- 172.21.15.116:0/1504820900 learned_addr learned my addr 172.21.15.116:0/1504820900 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 -- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 msgr2=0x560676201850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x560676201850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 -- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 msgr2=0x560676204480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x560676204480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x560676204480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 -- 172.21.15.116:0/1504820900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606760f5f20 con 0x5606761d40c0 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 0x560676201310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009370 tx=0x7ff1c8059340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0e9befec07a007 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1cdffb640 1 -- 172.21.15.116:0/1504820900 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8061210 con 0x5606761d40c0 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1cdffb640 1 -- 172.21.15.116:0/1504820900 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8056e00 con 0x5606761d40c0 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.463+0000 7ff1cdffb640 1 -- 172.21.15.116:0/1504820900 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c805db70 con 0x5606761d40c0 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1504820900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 msgr2=0x560676201310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 0x560676201310 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8009370 tx=0x7ff1c8059340 comp rx=0 tx=0).stop 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1504820900 shutdown_connections 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x560676201850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d40c0 0x560676201310 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1504820900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x560676204480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1504820900 >> 172.21.15.116:0/1504820900 conn(0x5606761d6330 msgr2=0x56067621eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1504820900 shutdown_connections 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1504820900 wait complete. 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 -- start start 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.464+0000 7ff1e11dc740 1 Processor -- start 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x5606762076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x5606761fd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 0x560676215940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x560676140c30 con 0x5606761fd7c0 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606760e5d30 con 0x5606762040a0 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x560675e14310 con 0x5606761d7c20 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1e11dc740 10 client.0 _ll_register_callbacks cb 0x5606760edd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 0x560676215940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 0x560676215940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39466/0 (socket says 172.21.15.116:39466) 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.465+0000 7ff1dcac9640 1 -- 172.21.15.116:0/1553712818 learned_addr learned my addr 172.21.15.116:0/1553712818 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x5606762076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x5606761fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1dcac9640 1 -- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 msgr2=0x5606762076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x5606762076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1dcac9640 1 -- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 msgr2=0x5606761fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x5606761fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1dcac9640 1 -- 172.21.15.116:0/1553712818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d806c120 con 0x5606761fd7c0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x5606761fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x5606762076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 0x560676215940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000d4b0 tx=0x7ff1d000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb22325b17cb847 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/1553712818 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00070d0 con 0x5606761fd7c0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/1553712818 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff1d0013dc0 con 0x5606761fd7c0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560675e406c0 con 0x5606761fd7c0 2024-02-18T11:48:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/1553712818 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d0007230 con 0x5606761fd7c0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 10 client.4863 fetch_fsmap learned FSMap version 8 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56067613f2b0 con 0x5606761fd7c0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/1553712818 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560675e406c0 con 0x5606761fd7c0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/1553712818 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff1d0007390 con 0x5606761fd7c0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 10 client.4863 fetch_fsmap finished waiting for FSMap version 8 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 10 client.4863 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 20 client.4863 populate_metadata read hostname 'smithi116' 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x5606761ec6e0 0x56067622f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 4 client.4863 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 --> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560675e91270 con 0x5606761ec6e0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.466+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x5606761ec6e0 0x56067622f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x5606761ec6e0 0x56067622f170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f13f85a9415dc1 server_cookie=6a55d9a82b3397e1 in_seq=0 out_seq=0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1b6ffd640 10 client.4863 ms_handle_connect on v2:172.21.15.116:6834/1897601775 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/1553712818 <== mds.0 v2:172.21.15.116:6834/1897601775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560675e91270 con 0x5606761ec6e0 2024-02-18T11:48:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1b6ffd640 10 client.4863 handle_command_reply: tid=0 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 client.4863 shutdown 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 msgr2=0x560676215940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 0x560676215940 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000d4b0 tx=0x7ff1d000d970 comp rx=0 tx=0).stop 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 shutdown_connections 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x5606761ec6e0 0x56067622f170 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761fd7c0 0x560676215940 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1b6ffd640 0 client.4863 ms_handle_reset on v2:172.21.15.116:6834/1897601775 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606762040a0 0x5606761fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.467+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/1553712818 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761d7c20 0x5606762076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.468+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 >> 172.21.15.116:0/1553712818 conn(0x5606761d40c0 msgr2=0x560676207c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.468+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 shutdown_connections 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.468+0000 7ff1e11dc740 1 -- 172.21.15.116:0/1553712818 wait complete. 2024-02-18T11:48:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.468+0000 7ff1e11dc740 20 client.4863 trim_cache size 0 max 16384 2024-02-18T11:48:07.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.483+0000 7ff1e11dc740 1 Processor -- start 2024-02-18T11:48:07.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.483+0000 7ff1e11dc740 1 -- start start 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761d7c20 0x5606761d9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761fd7c0 0x560676216a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 0x560676214840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606760c6490 con 0x5606762040a0 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x560676139cf0 con 0x5606761d7c20 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606761ecc00 con 0x5606761fd7c0 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 0x560676214840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 0x560676214840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39480/0 (socket says 172.21.15.116:39480) 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 -- 172.21.15.116:0/379303713 learned_addr learned my addr 172.21.15.116:0/379303713 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761d7c20 0x5606761d9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761fd7c0 0x560676216a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 -- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761fd7c0 msgr2=0x560676216a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761fd7c0 0x560676216a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 -- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761d7c20 msgr2=0x5606761d9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761d7c20 0x5606761d9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 -- 172.21.15.116:0/379303713 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606760dba20 con 0x5606762040a0 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.484+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 0x560676214840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000df10 tx=0x7ff1d0006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb4f51465b9bb1e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1cd7fa640 1 -- 172.21.15.116:0/379303713 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0007aa0 con 0x5606762040a0 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1cd7fa640 1 -- 172.21.15.116:0/379303713 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0007e90 con 0x5606762040a0 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1cd7fa640 1 -- 172.21.15.116:0/379303713 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0007310 con 0x5606762040a0 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 -- 172.21.15.116:0/379303713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 msgr2=0x560676214840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 0x560676214840 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000df10 tx=0x7ff1d0006000 comp rx=0 tx=0).stop 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 -- 172.21.15.116:0/379303713 shutdown_connections 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606762040a0 0x560676214840 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606761fd7c0 0x560676216a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/379303713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761d7c20 0x5606761d9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 -- 172.21.15.116:0/379303713 >> 172.21.15.116:0/379303713 conn(0x5606761d40c0 msgr2=0x560676200a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:07.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 -- 172.21.15.116:0/379303713 shutdown_connections 2024-02-18T11:48:07.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.485+0000 7ff1e11dc740 1 -- 172.21.15.116:0/379303713 wait complete. 2024-02-18T11:48:07.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.486+0000 7ff1e11dc740 1 -- start start 2024-02-18T11:48:07.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.486+0000 7ff1e11dc740 1 Processor -- start 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 0x560676214840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761fd7c0 0x560676214d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606762040a0 0x5606762314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x560676215400 con 0x5606761d7c20 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x56067623ed30 con 0x5606761fd7c0 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606761da370 con 0x5606762040a0 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 10 client.0 _ll_register_callbacks cb 0x7ff1c8054e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 0x560676214840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 0x560676214840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39496/0 (socket says 172.21.15.116:39496) 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 -- 172.21.15.116:0/2683611157 learned_addr learned my addr 172.21.15.116:0/2683611157 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:07.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1dcac9640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606762040a0 0x5606762314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761fd7c0 0x560676214d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 -- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606762040a0 msgr2=0x5606762314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606762040a0 0x5606762314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 -- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761fd7c0 msgr2=0x560676214d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761fd7c0 0x560676214d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 -- 172.21.15.116:0/2683611157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56067613f2b0 con 0x5606761d7c20 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1d7fff640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 0x560676214840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00098c0 tx=0x7ff1c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd586695180f4931 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/2683611157 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0040070 con 0x5606761d7c20 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/2683611157 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff1c005d470 con 0x5606761d7c20 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff1c8056e00 con 0x5606761d7c20 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/2683611157 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1c005d630 con 0x5606761d7c20 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 10 client.4869 fetch_fsmap learned FSMap version 8 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.487+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x560676101780 con 0x5606761d7c20 2024-02-18T11:48:07.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/2683611157 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8056e00 con 0x5606761d7c20 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/2683611157 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff1c003a420 con 0x5606761d7c20 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 10 client.4869 fetch_fsmap finished waiting for FSMap version 8 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 10 client.4869 resolve_mds: resolved name 'a' to daemon mds.a 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 20 client.4869 populate_metadata read hostname 'smithi116' 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x560676231a30 0x560676231e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 4 client.4869 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 --> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560675e7d610 con 0x560676231a30 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x560676231a30 0x560676231e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1b6ffd640 10 client.4869 ms_handle_connect on v2:172.21.15.116:6834/1897601775 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1d77fe640 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x560676231a30 0x560676231e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5163f9caafba2e1e server_cookie=ca7ad301617e231e in_seq=0 out_seq=0 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1b6ffd640 1 -- 172.21.15.116:0/2683611157 <== mds.0 v2:172.21.15.116:6834/1897601775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x560675e7d610 con 0x560676231a30 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1b6ffd640 10 client.4869 handle_command_reply: tid=0 2024-02-18T11:48:07.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 1 client.4869 shutdown 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 msgr2=0x560676214840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.488+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 0x560676214840 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00098c0 tx=0x7ff1c0004730 comp rx=0 tx=0).stop 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 shutdown_connections 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:6834/1897601775,v1:172.21.15.116:6836/1897601775] conn(0x560676231a30 0x560676231e10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x5606762040a0 0x5606762314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5606761fd7c0 0x560676214d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1b6ffd640 0 client.4869 ms_handle_reset on v2:172.21.15.116:6834/1897601775 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 --2- 172.21.15.116:0/2683611157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606761d7c20 0x560676214840 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 >> 172.21.15.116:0/2683611157 conn(0x5606761d40c0 msgr2=0x5606761d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 shutdown_connections 2024-02-18T11:48:07.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 1 -- 172.21.15.116:0/2683611157 wait complete. 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1e11dc740 20 client.4869 trim_cache size 0 max 16384 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "82e3e917-bb19-48fa-a66a-62df77a0f413", 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4281, 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-02-18T11:48:07.489 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 32.971999664000002, 2024-02-18T11:48:07.490 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 8, 2024-02-18T11:48:07.490 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 24, 2024-02-18T11:48:07.490 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 24, 2024-02-18T11:48:07.490 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 39.573214079000003 2024-02-18T11:48:07.490 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff1b806bc70 msgr2=0x7ff1b806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff1b806bc70 0x7ff1b806e130 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0036000 tx=0x7ff1c0008030 comp rx=0 tx=0).stop 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 msgr2=0x7ff1d815e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d815e6c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b3f0 tx=0x7ff1c800b8b0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 shutdown_connections 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff1b806bc70 0x7ff1b806e130 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1d815fca0 0x7ff1d8156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1d815f8c0 0x7ff1d815e6c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 --2- 172.21.15.116:0/3637112053 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff1d815b560 0x7ff1d815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 >> 172.21.15.116:0/3637112053 conn(0x7ff1d80b31f0 msgr2=0x7ff1d806f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 shutdown_connections 2024-02-18T11:48:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:07.489+0000 7ff1de52c640 1 -- 172.21.15.116:0/3637112053 wait complete. 2024-02-18T11:48:07.493 INFO:teuthology.run_tasks:Running task kclient... 2024-02-18T11:48:07.506 INFO:tasks.kclient:Mounting kernel clients... 2024-02-18T11:48:07.506 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-02-18T11:48:07.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.506 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-02-18T11:48:07.521 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-18T11:48:07.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.522 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-18T11:48:07.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.590 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-02-18T11:48:07.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.699 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-02-18T11:48:07.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.715 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-02-18T11:48:07.787 INFO:teuthology.orchestra.run.smithi116.stderr:Cannot find device "ceph-brx" 2024-02-18T11:48:07.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:48:07.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.789 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-02-18T11:48:07.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-02-18T11:48:07.881 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2024-02-18T11:48:07.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:48:07.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.883 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-02-18T11:48:07.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:07.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-02-18T11:48:08.013 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2024-02-18T11:48:08.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T11:48:08.015 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-02-18T11:48:08.015 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-18T11:48:08.015 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:08.015 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-18T11:48:08.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.123+0000 7ff77c440640 1 Processor -- start 2024-02-18T11:48:08.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.123+0000 7ff77c440640 1 -- start start 2024-02-18T11:48:08.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.124+0000 7ff77c440640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff774155e30 0x7ff77415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:08.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.124+0000 7ff77c440640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff77415bdc0 0x7ff77415c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:08.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.124+0000 7ff77c440640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77406d6f0 con 0x7ff7740b1740 2024-02-18T11:48:08.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.124+0000 7ff77c440640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77415c880 con 0x7ff774155e30 2024-02-18T11:48:08.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.124+0000 7ff77c440640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77415cb60 con 0x7ff77415bdc0 2024-02-18T11:48:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff7799db640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff77415bdc0 0x7ff77415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a1dc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff774155e30 0x7ff77415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a9dd640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff7740b1740 0x7ff77406db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:49744/0 (socket says 172.21.15.116:49744) 2024-02-18T11:48:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a9dd640 1 -- 172.21.15.116:0/1022810090 learned_addr learned my addr 172.21.15.116:0/1022810090 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a1dc640 1 -- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff77415bdc0 msgr2=0x7ff77415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:08.120 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a1dc640 1 --2- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff77415bdc0 0x7ff77415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a1dc640 1 -- 172.21.15.116:0/1022810090 >> v1:172.21.15.143:6789/0 conn(0x7ff7740b1740 legacy=0x7ff77406db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:08.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff77a1dc640 1 -- 172.21.15.116:0/1022810090 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77406d380 con 0x7ff774155e30 2024-02-18T11:48:08.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.125+0000 7ff7799db640 1 --2- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff77415bdc0 0x7ff77415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:08.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.126+0000 7ff77a1dc640 1 --2- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff77415cf80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff770009d90 tx=0x7ff77002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1341dcd4c50b29fa server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.126+0000 7ff7791da640 1 -- 172.21.15.116:0/1022810090 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77002cc50 con 0x7ff774155e30 2024-02-18T11:48:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.126+0000 7ff7791da640 1 -- 172.21.15.116:0/1022810090 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77002cdb0 con 0x7ff774155e30 2024-02-18T11:48:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.126+0000 7ff7791da640 1 -- 172.21.15.116:0/1022810090 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770034660 con 0x7ff774155e30 2024-02-18T11:48:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 -- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 msgr2=0x7ff77415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 --2- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff77415cf80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff770009d90 tx=0x7ff77002bea0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 -- 172.21.15.116:0/1022810090 shutdown_connections 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 --2- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff77415bdc0 0x7ff77415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 --2- 172.21.15.116:0/1022810090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff77415cf80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 -- 172.21.15.116:0/1022810090 >> 172.21.15.116:0/1022810090 conn(0x7ff774066f20 msgr2=0x7ff77406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 -- 172.21.15.116:0/1022810090 shutdown_connections 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.127+0000 7ff77c440640 1 -- 172.21.15.116:0/1022810090 wait complete. 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.128+0000 7ff77c440640 1 Processor -- start 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.128+0000 7ff77c440640 1 -- start start 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77c440640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff7740b1740 0x7ff774141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77c440640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff7741484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77c440640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 0x7ff77414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:08.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77c440640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77414f110 con 0x7ff77415bdc0 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77c440640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77414f3f0 con 0x7ff774155e30 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77c440640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77414f6d0 con 0x7ff7740b1740 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 0x7ff77414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 0x7ff77414e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39512/0 (socket says 172.21.15.116:39512) 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 -- 172.21.15.116:0/3861890784 learned_addr learned my addr 172.21.15.116:0/3861890784 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a1dc640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff7740b1740 0x7ff774141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff7799db640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff7741484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 -- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff7740b1740 msgr2=0x7ff774141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff7740b1740 0x7ff774141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 -- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 msgr2=0x7ff7741484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff7741484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.128 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 -- 172.21.15.116:0/3861890784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff764009400 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a1dc640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff7740b1740 0x7ff774141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff7799db640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff7741484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.129+0000 7ff77a9dd640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 0x7ff77414e3b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c420 tx=0x7ff76c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.130+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c005ce0 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.130+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c005e40 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.130+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0072d0 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.130+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff770009a70 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.130+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77414fc90 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.131+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77414fe80 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.131+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff76c0074f0 con 0x7ff77415bdc0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.131+0000 7ff76affd640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff75806bc70 0x7ff75806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.132+0000 7ff77a1dc640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff75806bc70 0x7ff75806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:08.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.132+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff76c089f70 con 0x7ff77415bdc0 2024-02-18T11:48:08.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.132+0000 7ff77a1dc640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff75806bc70 0x7ff75806e130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff770036000 tx=0x7ff770030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:08.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.136+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff76c0bd8c0 con 0x7ff77415bdc0 2024-02-18T11:48:08.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.342+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff774074520 con 0x7ff77415bdc0 2024-02-18T11:48:08.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.342+0000 7ff76affd640 1 -- 172.21.15.116:0/3861890784 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff76c017020 con 0x7ff77415bdc0 2024-02-18T11:48:08.336 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff75806bc70 msgr2=0x7ff75806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff75806bc70 0x7ff75806e130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff770036000 tx=0x7ff770030000 comp rx=0 tx=0).stop 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 msgr2=0x7ff77414e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 0x7ff77414e3b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00c420 tx=0x7ff76c00c8e0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 shutdown_connections 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff75806bc70 0x7ff75806e130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff77415bdc0 0x7ff77414e3b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff774155e30 0x7ff7741484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 --2- 172.21.15.116:0/3861890784 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff7740b1740 0x7ff774141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:08.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 >> 172.21.15.116:0/3861890784 conn(0x7ff774066f20 msgr2=0x7ff774073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:08.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 shutdown_connections 2024-02-18T11:48:08.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:08.344+0000 7ff77c440640 1 -- 172.21.15.116:0/3861890784 wait complete. 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:48:08.349 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:48:08.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.350 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:08.365 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42746sec preferred_lft 42746sec 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:08.366 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-18T11:48:08.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-18T11:48:08.367 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:08.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.474 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:48:08.499 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-02-18T11:48:08.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.500 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-02-18T11:48:08.556 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-02-18T11:48:08.556 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:48:08.556 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-18T11:48:08.556 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-18T11:48:08.556 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:48:08.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.556 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:08.557 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:08.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-18T11:48:08.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-18T11:48:08.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-18T11:48:08.557 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:08.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.653 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-02-18T11:48:08.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.668 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-02-18T11:48:08.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.723 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:08.723 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:08.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-18T11:48:08.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-18T11:48:08.723 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:08.805 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-18T11:48:08.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.805 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-18T11:48:08.806 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:08.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:08.931 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:08.931 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:08.931 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-02-18T11:48:08.931 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2024-02-18T11:48:08.931 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:08.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:08.973 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:09.028 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:48:09.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:09.028 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:09.083 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-18T11:48:09.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.083 DEBUG:teuthology.orchestra.run.smithi116:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:09.167 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:09.167 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-18T11:48:09.167 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:09.271 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:09.272 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-18T11:48:09.272 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:09.272 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:09.272 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:09.276 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790" 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790:/ 2024-02-18T11:48:09.277 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413" will pass to kernel 2024-02-18T11:48:09.278 INFO:teuthology.orchestra.run.smithi116.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-18T11:48:09.278 INFO:teuthology.orchestra.run.smithi116.stdout: You just mounted a file system that supports labels which does not 2024-02-18T11:48:09.278 INFO:teuthology.orchestra.run.smithi116.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-18T11:48:09.278 INFO:teuthology.orchestra.run.smithi116.stdout: applications will generate AVC messages and not be allowed access to 2024-02-18T11:48:09.278 INFO:teuthology.orchestra.run.smithi116.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-18T11:48:09.279 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-18T11:48:09.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.279 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:09.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.306 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:09.362 INFO:teuthology.orchestra.run.smithi116.stdout:client4672 2024-02-18T11:48:09.363 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:09.363 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:09.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.419 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:09.475 INFO:teuthology.orchestra.run.smithi116.stdout:client4672 2024-02-18T11:48:09.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.475 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:09.531 INFO:teuthology.orchestra.run.smithi116.stdout:client4672 2024-02-18T11:48:09.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.531 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/82e3e917-bb19-48fa-a66a-62df77a0f413.client4672/status') 2024-02-18T11:48:09.597 INFO:teuthology.orchestra.run.smithi116.stdout:instance: client.4672 (3)172.21.15.116:0/4213698815 2024-02-18T11:48:09.598 INFO:teuthology.orchestra.run.smithi116.stdout:blocklisted: false 2024-02-18T11:48:09.598 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2024-02-18T11:48:09.598 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2024-02-18T11:48:09.598 INFO:teuthology.orchestra.run.smithi116.stderr:71 bytes copied, 3.7384e-05 s, 1.9 MB/s 2024-02-18T11:48:09.599 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-02-18T11:48:09.599 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-18T11:48:09.599 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-02-18T11:48:09.600 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-18T11:48:09.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.747+0000 7fe14ca2e640 1 Processor -- start 2024-02-18T11:48:09.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.747+0000 7fe14ca2e640 1 -- start start 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14ca2e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1480d1080 0x7fe1480b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14ca2e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe148161850 0x7fe1480ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14ca2e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1480ce780 con 0x7fe14815dc60 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14ca2e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1480d3730 con 0x7fe1480d1080 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14ca2e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1480cf250 con 0x7fe148161850 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe145d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1480d1080 0x7fe1480b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14659c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe14815dc60 0x7fe1480d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59722/0 (socket says 172.21.15.116:59722) 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.748+0000 7fe14659c640 1 -- 172.21.15.116:0/1464980090 learned_addr learned my addr 172.21.15.116:0/1464980090 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:09.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe146d9d640 1 --2- 172.21.15.116:0/1464980090 >> v2:172.21.15.143:3300/0 conn(0x7fe148161850 0x7fe1480ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:09.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe145d9b640 1 -- 172.21.15.116:0/1464980090 >> v2:172.21.15.143:3300/0 conn(0x7fe148161850 msgr2=0x7fe1480ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:09.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe145d9b640 1 --2- 172.21.15.116:0/1464980090 >> v2:172.21.15.143:3300/0 conn(0x7fe148161850 0x7fe1480ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe145d9b640 1 -- 172.21.15.116:0/1464980090 >> v1:172.21.15.116:6789/0 conn(0x7fe14815dc60 legacy=0x7fe1480d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:09.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe145d9b640 1 -- 172.21.15.116:0/1464980090 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480ce9b0 con 0x7fe1480d1080 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe145d9b640 1 --2- 172.21.15.116:0/1464980090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1480d1080 0x7fe1480b6c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe1480b5480 tx=0x7fe13c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37502b95837ad5c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.749+0000 7fe14559a640 1 -- 172.21.15.116:0/1464980090 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c031830 con 0x7fe1480d1080 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14559a640 1 -- 172.21.15.116:0/1464980090 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c031990 con 0x7fe1480d1080 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14559a640 1 -- 172.21.15.116:0/1464980090 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c031d20 con 0x7fe1480d1080 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1464980090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1480d1080 msgr2=0x7fe1480b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1464980090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1480d1080 0x7fe1480b6c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe1480b5480 tx=0x7fe13c02b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1464980090 shutdown_connections 2024-02-18T11:48:09.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1464980090 >> v2:172.21.15.143:3300/0 conn(0x7fe148161850 0x7fe1480ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1464980090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1480d1080 0x7fe1480b6c10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1464980090 >> 172.21.15.116:0/1464980090 conn(0x7fe1480b5910 msgr2=0x7fe1480b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:09.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1464980090 shutdown_connections 2024-02-18T11:48:09.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.750+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1464980090 wait complete. 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.751+0000 7fe14ca2e640 1 Processor -- start 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.751+0000 7fe14ca2e640 1 -- start start 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14ca2e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1480d1080 0x7fe1481f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14ca2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 0x7fe1480c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14ca2e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe148161850 0x7fe1480c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14ca2e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14815e5c0 con 0x7fe14815dc60 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14ca2e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481f3850 con 0x7fe1480d1080 2024-02-18T11:48:09.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14ca2e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe148160dd0 con 0x7fe148161850 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe145d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 0x7fe1480c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe145d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 0x7fe1480c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39534/0 (socket says 172.21.15.116:39534) 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe145d9b640 1 -- 172.21.15.116:0/1749832641 learned_addr learned my addr 172.21.15.116:0/1749832641 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe14659c640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1480d1080 0x7fe1481f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.752+0000 7fe146d9d640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe148161850 0x7fe1480c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe145d9b640 1 -- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe148161850 msgr2=0x7fe1480c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe145d9b640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe148161850 0x7fe1480c7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe145d9b640 1 -- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1480d1080 msgr2=0x7fe1481f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe145d9b640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1480d1080 0x7fe1481f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe145d9b640 1 -- 172.21.15.116:0/1749832641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c009400 con 0x7fe14815dc60 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe145d9b640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 0x7fe1480c7510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe13c000c00 tx=0x7fe13c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c03aa60 con 0x7fe14815dc60 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c03abe0 con 0x7fe14815dc60 2024-02-18T11:48:09.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0443f0 con 0x7fe14815dc60 2024-02-18T11:48:09.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14815e830 con 0x7fe14815dc60 2024-02-18T11:48:09.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.753+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1480d42d0 con 0x7fe14815dc60 2024-02-18T11:48:09.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.754+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe148067840 con 0x7fe14815dc60 2024-02-18T11:48:09.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.754+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe13c04d450 con 0x7fe14815dc60 2024-02-18T11:48:09.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.755+0000 7fe136ffd640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe12806bc70 0x7fe12806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:09.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.755+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe13c0acbc0 con 0x7fe14815dc60 2024-02-18T11:48:09.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.755+0000 7fe14659c640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe12806bc70 0x7fe12806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:09.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.755+0000 7fe14659c640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe12806bc70 0x7fe12806e130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1380012e0 tx=0x7fe138009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:09.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.759+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe13c0770f0 con 0x7fe14815dc60 2024-02-18T11:48:09.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.970+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe14815e420 con 0x7fe14815dc60 2024-02-18T11:48:09.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.970+0000 7fe136ffd640 1 -- 172.21.15.116:0/1749832641 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe14815e420 con 0x7fe14815dc60 2024-02-18T11:48:09.964 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-18T11:48:09.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe12806bc70 msgr2=0x7fe12806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe12806bc70 0x7fe12806e130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1380012e0 tx=0x7fe138009040 comp rx=0 tx=0).stop 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 msgr2=0x7fe1480c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 0x7fe1480c7510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe13c000c00 tx=0x7fe13c0046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 shutdown_connections 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe12806bc70 0x7fe12806e130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe148161850 0x7fe1480c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14815dc60 0x7fe1480c7510 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 --2- 172.21.15.116:0/1749832641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1480d1080 0x7fe1481f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 >> 172.21.15.116:0/1749832641 conn(0x7fe1480b5910 msgr2=0x7fe148073630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 shutdown_connections 2024-02-18T11:48:09.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:09.972+0000 7fe14ca2e640 1 -- 172.21.15.116:0/1749832641 wait complete. 2024-02-18T11:48:09.976 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-18T11:48:09.976 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:48:09.976 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-02-18T11:48:09.977 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-02-18T11:48:09.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-02-18T11:48:09.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-18T11:48:09.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-18T11:48:09.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:48:09.977 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-02-18T11:48:09.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.977 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-02-18T11:48:09.993 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42744sec preferred_lft 42744sec 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 8a:a8:c5:f8:ac:12 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-18T11:48:09.994 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::6c0a:43ff:fedd:aff6/64 scope link 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 8a:a8:c5:f8:ac:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::88a8:c5ff:fef8:ac12/64 scope link tentative 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:09.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:09.995 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:48:10.058 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-02-18T11:48:10.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.060 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-02-18T11:48:10.116 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-02-18T11:48:10.116 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:48:10.116 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-18T11:48:10.117 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-18T11:48:10.117 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:48:10.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.117 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:10.117 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:10.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-18T11:48:10.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-18T11:48:10.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-18T11:48:10.117 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:10.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.212 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-02-18T11:48:10.227 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-18T11:48:10.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.228 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-02-18T11:48:10.283 INFO:teuthology.orchestra.run.smithi116.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-02-18T11:48:10.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.283 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:10.284 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:10.284 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-18T11:48:10.284 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-02-18T11:48:10.284 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:10.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.365 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-02-18T11:48:10.431 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 22:80:1c:cc:7b:7b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2080:1cff:fecc:7b7b/64 scope link 2024-02-18T11:48:10.432 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.434 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 22:80:1c:cc:7b:7b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-18T11:48:10.501 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.502 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2080:1cff:fecc:7b7b/64 scope link 2024-02-18T11:48:10.502 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:10.503 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-02-18T11:48:10.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.503 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:10.503 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:10.503 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-02-18T11:48:10.503 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-02-18T11:48:10.503 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-02-18T11:48:10.503 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-02-18T11:48:10.504 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-02-18T11:48:10.504 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:10.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.630 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:10.630 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:10.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2024-02-18T11:48:10.630 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.1 master ceph-brx 2024-02-18T11:48:10.630 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:10.672 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:10.672 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-02-18T11:48:10.728 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-02-18T11:48:10.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:10.728 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-02-18T11:48:10.782 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-18T11:48:10.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.783 DEBUG:teuthology.orchestra.run.smithi116:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-02-18T11:48:10.867 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-02-18T11:48:10.867 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-02-18T11:48:10.867 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:10.875 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-02-18T11:48:10.875 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-02-18T11:48:10.875 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:10.875 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:10.876 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790" 2024-02-18T11:48:10.886 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790:/ 2024-02-18T11:48:10.887 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413" will pass to kernel 2024-02-18T11:48:10.887 INFO:teuthology.orchestra.run.smithi116.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-02-18T11:48:10.887 INFO:teuthology.orchestra.run.smithi116.stdout: You just mounted a file system that supports labels which does not 2024-02-18T11:48:10.887 INFO:teuthology.orchestra.run.smithi116.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-18T11:48:10.887 INFO:teuthology.orchestra.run.smithi116.stdout: applications will generate AVC messages and not be allowed access to 2024-02-18T11:48:10.887 INFO:teuthology.orchestra.run.smithi116.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-18T11:48:10.889 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-18T11:48:10.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-02-18T11:48:10.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:10.916 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:10.972 INFO:teuthology.orchestra.run.smithi116.stdout:client4884 2024-02-18T11:48:10.972 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:10.972 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:11.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:11.027 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:11.084 INFO:teuthology.orchestra.run.smithi116.stdout:client4884 2024-02-18T11:48:11.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:11.084 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:11.140 INFO:teuthology.orchestra.run.smithi116.stdout:client4884 2024-02-18T11:48:11.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:11.141 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/82e3e917-bb19-48fa-a66a-62df77a0f413.client4884/status') 2024-02-18T11:48:11.206 INFO:teuthology.orchestra.run.smithi116.stdout:instance: client.4884 (3)192.168.0.2:0/3627651131 2024-02-18T11:48:11.206 INFO:teuthology.orchestra.run.smithi116.stdout:blocklisted: false 2024-02-18T11:48:11.206 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2024-02-18T11:48:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2024-02-18T11:48:11.207 INFO:teuthology.orchestra.run.smithi116.stderr:69 bytes copied, 1.9419e-05 s, 3.6 MB/s 2024-02-18T11:48:11.208 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-02-18T11:48:11.208 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-18T11:48:11.208 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-18T11:48:11.208 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-18T11:48:11.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.355+0000 7f5501adf640 1 Processor -- start 2024-02-18T11:48:11.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.355+0000 7f5501adf640 1 -- start start 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.355+0000 7f5501adf640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54fc161f50 0x7f54fc0b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.355+0000 7f5501adf640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc15e1a0 con 0x7f54fc15ddc0 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.355+0000 7f5501adf640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0b96a0 con 0x7f54fc0b6b00 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.355+0000 7f5501adf640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc1623e0 con 0x7f54fc161f50 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54faffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54fc161f50 0x7f54fc0b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54fb7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f54fc15ddc0 0x7f54fc161840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59724/0 (socket says 172.21.15.116:59724) 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54fb7fe640 1 -- 172.21.15.116:0/2169266367 learned_addr learned my addr 172.21.15.116:0/2169266367 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54faffd640 1 -- 172.21.15.116:0/2169266367 >> v1:172.21.15.143:6789/0 conn(0x7f54fc0b6b00 legacy=0x7f54fc0d2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54faffd640 1 -- 172.21.15.116:0/2169266367 >> v1:172.21.15.116:6789/0 conn(0x7f54fc15ddc0 legacy=0x7f54fc161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54faffd640 1 -- 172.21.15.116:0/2169266367 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0d1c60 con 0x7f54fc161f50 2024-02-18T11:48:11.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.356+0000 7f54fa7fc640 1 -- 172.21.15.116:0/2169266367 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540253739 0 0) 0x7f54ec001490 con 0x7f54fc15ddc0 2024-02-18T11:48:11.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f54faffd640 1 --2- 172.21.15.116:0/2169266367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc161f50 0x7f54fc0b57d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097a0 tx=0x7f54f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4e94a602c9f676 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:11.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f54fa7fc640 1 -- 172.21.15.116:0/2169266367 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0030610 con 0x7f54fc161f50 2024-02-18T11:48:11.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f54fa7fc640 1 -- 172.21.15.116:0/2169266367 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f0030770 con 0x7f54fc161f50 2024-02-18T11:48:11.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f54fa7fc640 1 -- 172.21.15.116:0/2169266367 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0030b00 con 0x7f54fc161f50 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 -- 172.21.15.116:0/2169266367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc161f50 msgr2=0x7f54fc0b57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 --2- 172.21.15.116:0/2169266367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc161f50 0x7f54fc0b57d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54f00097a0 tx=0x7f54f002b960 comp rx=0 tx=0).stop 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 -- 172.21.15.116:0/2169266367 shutdown_connections 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 --2- 172.21.15.116:0/2169266367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc161f50 0x7f54fc0b57d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 -- 172.21.15.116:0/2169266367 >> 172.21.15.116:0/2169266367 conn(0x7f54fc0d0870 msgr2=0x7f54fc0d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 -- 172.21.15.116:0/2169266367 shutdown_connections 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.357+0000 7f5501adf640 1 -- 172.21.15.116:0/2169266367 wait complete. 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.358+0000 7f5501adf640 1 Processor -- start 2024-02-18T11:48:11.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 -- start start 2024-02-18T11:48:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b6b00 0x7f54fc0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 0x7f54fc0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f54fc148e40 0x7f54fc157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cc030 con 0x7f54fc15ddc0 2024-02-18T11:48:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cc2b0 con 0x7f54fc0b6b00 2024-02-18T11:48:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f5501adf640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cc590 con 0x7f54fc148e40 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f54faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 0x7f54fc0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f54faffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 0x7f54fc0c6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39558/0 (socket says 172.21.15.116:39558) 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.359+0000 7f54faffd640 1 -- 172.21.15.116:0/775497973 learned_addr learned my addr 172.21.15.116:0/775497973 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54fb7fe640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b6b00 0x7f54fc0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54fbfff640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f54fc148e40 0x7f54fc157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54faffd640 1 -- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f54fc148e40 msgr2=0x7f54fc157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54faffd640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f54fc148e40 0x7f54fc157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54faffd640 1 -- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b6b00 msgr2=0x7f54fc0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54faffd640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b6b00 0x7f54fc0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54faffd640 1 -- 172.21.15.116:0/775497973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0009400 con 0x7f54fc15ddc0 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54fb7fe640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b6b00 0x7f54fc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54fbfff640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f54fc148e40 0x7f54fc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54faffd640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 0x7f54fc0c6490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54f00098c0 tx=0x7f54f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f003e070 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f0039a10 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc0cc870 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0039da0 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.360+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc0cce30 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.361+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc0cf930 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.361+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f54f004b400 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.362+0000 7f54e3fff640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f54e806bca0 0x7f54e806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.362+0000 7f54fb7fe640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f54e806bca0 0x7f54e806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.362+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f54f00ad0f0 con 0x7f54fc15ddc0 2024-02-18T11:48:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.362+0000 7f54fb7fe640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f54e806bca0 0x7f54e806e160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54ec007fd0 tx=0x7f54ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:11.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.366+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54f00776b0 con 0x7f54fc15ddc0 2024-02-18T11:48:11.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.575+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f54fc0ce670 con 0x7f54fc15ddc0 2024-02-18T11:48:11.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.575+0000 7f54e3fff640 1 -- 172.21.15.116:0/775497973 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f54f007ad70 con 0x7f54fc15ddc0 2024-02-18T11:48:11.569 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-18T11:48:11.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f54e806bca0 msgr2=0x7f54e806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f54e806bca0 0x7f54e806e160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f54ec007fd0 tx=0x7f54ec009040 comp rx=0 tx=0).stop 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 msgr2=0x7f54fc0c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 0x7f54fc0c6490 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54f00098c0 tx=0x7f54f00046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 shutdown_connections 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f54e806bca0 0x7f54e806e160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f54fc148e40 0x7f54fc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54fc15ddc0 0x7f54fc0c6490 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 --2- 172.21.15.116:0/775497973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b6b00 0x7f54fc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 >> 172.21.15.116:0/775497973 conn(0x7f54fc0d0870 msgr2=0x7f54fc0b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 shutdown_connections 2024-02-18T11:48:11.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:11.577+0000 7f5501adf640 1 -- 172.21.15.116:0/775497973 wait complete. 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-18T11:48:11.582 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-18T11:48:11.583 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:48:11.583 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-18T11:48:11.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:11.583 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:11.603 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42740sec preferred_lft 42740sec 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:11.604 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-18T11:48:11.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-18T11:48:11.605 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:12.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.035 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:48:12.064 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-02-18T11:48:12.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.065 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-02-18T11:48:12.131 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-02-18T11:48:12.132 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:48:12.132 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-18T11:48:12.132 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-18T11:48:12.132 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:48:12.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.132 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:12.132 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:12.132 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-18T11:48:12.132 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-18T11:48:12.133 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-18T11:48:12.133 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:12.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.408 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-02-18T11:48:12.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.422 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-02-18T11:48:12.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.477 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:12.477 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:12.477 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-18T11:48:12.477 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-18T11:48:12.477 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:12.559 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-18T11:48:12.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-18T11:48:12.560 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:12.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.720 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:12.720 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:12.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-02-18T11:48:12.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2024-02-18T11:48:12.721 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:12.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:12.763 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-18T11:48:12.818 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-18T11:48:12.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:12.818 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-18T11:48:12.873 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-18T11:48:12.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:12.873 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-18T11:48:12.958 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-18T11:48:12.958 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-02-18T11:48:12.958 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-02-18T11:48:13.058 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-18T11:48:13.058 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-02-18T11:48:13.058 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-02-18T11:48:13.059 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:13.059 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790" 2024-02-18T11:48:13.064 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790:/ 2024-02-18T11:48:13.065 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413" will pass to kernel 2024-02-18T11:48:13.065 INFO:teuthology.orchestra.run.smithi143.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-02-18T11:48:13.065 INFO:teuthology.orchestra.run.smithi143.stdout: You just mounted a file system that supports labels which does not 2024-02-18T11:48:13.065 INFO:teuthology.orchestra.run.smithi143.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-18T11:48:13.065 INFO:teuthology.orchestra.run.smithi143.stdout: applications will generate AVC messages and not be allowed access to 2024-02-18T11:48:13.065 INFO:teuthology.orchestra.run.smithi143.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-18T11:48:13.066 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-18T11:48:13.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-18T11:48:13.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.093 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:13.150 INFO:teuthology.orchestra.run.smithi143.stdout:client4684 2024-02-18T11:48:13.150 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:13.150 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:13.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.206 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:13.262 INFO:teuthology.orchestra.run.smithi143.stdout:client4684 2024-02-18T11:48:13.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.263 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:13.319 INFO:teuthology.orchestra.run.smithi143.stdout:client4684 2024-02-18T11:48:13.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.319 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/82e3e917-bb19-48fa-a66a-62df77a0f413.client4684/status') 2024-02-18T11:48:13.385 INFO:teuthology.orchestra.run.smithi143.stdout:instance: client.4684 (3)192.168.0.1:0/83664823 2024-02-18T11:48:13.386 INFO:teuthology.orchestra.run.smithi143.stdout:blocklisted: false 2024-02-18T11:48:13.386 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records in 2024-02-18T11:48:13.386 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records out 2024-02-18T11:48:13.386 INFO:teuthology.orchestra.run.smithi143.stderr:67 bytes copied, 2.2116e-05 s, 3.0 MB/s 2024-02-18T11:48:13.387 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-02-18T11:48:13.387 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-18T11:48:13.387 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-02-18T11:48:13.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-18T11:48:13.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.506+0000 7f99c776c640 1 Processor -- start 2024-02-18T11:48:13.500 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c776c640 1 -- start start 2024-02-18T11:48:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c776c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99c00b0750 0x7f99c00b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c776c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99c00d0730 0x7f99c00cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c776c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00b1070 con 0x7f99c00d0b90 2024-02-18T11:48:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c776c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00ccb70 con 0x7f99c00d0730 2024-02-18T11:48:13.501 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c776c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00cf970 con 0x7f99c00b0750 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c4d07640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99c00d0730 0x7f99c00cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.507+0000 7f99c5d09640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f99c00d0b90 0x7f99c00cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59738/0 (socket says 172.21.15.116:59738) 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c4d07640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99c00d0730 0x7f99c00cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39570/0 (socket says 172.21.15.116:39570) 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c5d09640 1 -- 172.21.15.116:0/1547425630 learned_addr learned my addr 172.21.15.116:0/1547425630 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c5508640 1 --2- 172.21.15.116:0/1547425630 >> v2:172.21.15.143:3300/0 conn(0x7f99c00b0750 0x7f99c00b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c4d07640 1 -- 172.21.15.116:0/1547425630 >> v2:172.21.15.143:3300/0 conn(0x7f99c00b0750 msgr2=0x7f99c00b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c4d07640 1 --2- 172.21.15.116:0/1547425630 >> v2:172.21.15.143:3300/0 conn(0x7f99c00b0750 0x7f99c00b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c4d07640 1 -- 172.21.15.116:0/1547425630 >> v1:172.21.15.116:6789/0 conn(0x7f99c00d0b90 legacy=0x7f99c00cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99c4d07640 1 -- 172.21.15.116:0/1547425630 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00cfc50 con 0x7f99c00d0730 2024-02-18T11:48:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.508+0000 7f99b7fff640 1 -- 172.21.15.116:0/1547425630 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275385881 0 0) 0x7f99c00b1070 con 0x7f99c00d0b90 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c4d07640 1 --2- 172.21.15.116:0/1547425630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c00d0730 0x7f99c00cbe40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f99c0072d20 tx=0x7f99b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd6dc6408e5afcc server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99b7fff640 1 -- 172.21.15.116:0/1547425630 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0031830 con 0x7f99c00d0730 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99b7fff640 1 -- 172.21.15.116:0/1547425630 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b0031990 con 0x7f99c00d0730 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99b7fff640 1 -- 172.21.15.116:0/1547425630 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0031d20 con 0x7f99c00d0730 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 -- 172.21.15.116:0/1547425630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c00d0730 msgr2=0x7f99c00cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 --2- 172.21.15.116:0/1547425630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c00d0730 0x7f99c00cbe40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f99c0072d20 tx=0x7f99b002b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 -- 172.21.15.116:0/1547425630 shutdown_connections 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 --2- 172.21.15.116:0/1547425630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c00d0730 0x7f99c00cbe40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 --2- 172.21.15.116:0/1547425630 >> v2:172.21.15.143:3300/0 conn(0x7f99c00b0750 0x7f99c00b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 -- 172.21.15.116:0/1547425630 >> 172.21.15.116:0/1547425630 conn(0x7f99c00b3380 msgr2=0x7f99c00b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 -- 172.21.15.116:0/1547425630 shutdown_connections 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.509+0000 7f99c776c640 1 -- 172.21.15.116:0/1547425630 wait complete. 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 Processor -- start 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 -- start start 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99c00b0750 0x7f99c0141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00d0730 0x7f99c01519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 0x7f99c0151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00ca8b0 con 0x7f99c015cc30 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0141a00 con 0x7f99c00d0730 2024-02-18T11:48:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c776c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c9f50 con 0x7f99c00b0750 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c5d09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 0x7f99c0151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c5d09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 0x7f99c0151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39584/0 (socket says 172.21.15.116:39584) 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.511+0000 7f99c5d09640 1 -- 172.21.15.116:0/467703979 learned_addr learned my addr 172.21.15.116:0/467703979 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5508640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99c00b0750 0x7f99c0141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c4d07640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00d0730 0x7f99c01519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5d09640 1 -- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99c00b0750 msgr2=0x7f99c0141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5d09640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99c00b0750 0x7f99c0141cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5d09640 1 -- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00d0730 msgr2=0x7f99c01519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5d09640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00d0730 0x7f99c01519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5d09640 1 -- 172.21.15.116:0/467703979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0009400 con 0x7f99c015cc30 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c5d09640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 0x7f99c0151f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c410 tx=0x7f99b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80074e0 con 0x7f99c015cc30 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b8007660 con 0x7f99c015cc30 2024-02-18T11:48:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c00ca1f0 con 0x7f99c015cc30 2024-02-18T11:48:13.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004a20 con 0x7f99c015cc30 2024-02-18T11:48:13.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.512+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c0154cf0 con 0x7f99c015cc30 2024-02-18T11:48:13.509 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.513+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0067bb0 con 0x7f99c015cc30 2024-02-18T11:48:13.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.514+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f99b8004b80 con 0x7f99c015cc30 2024-02-18T11:48:13.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.514+0000 7f99b5ffb640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99a406bc70 0x7f99a406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:13.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.514+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f99b808a700 con 0x7f99c015cc30 2024-02-18T11:48:13.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.514+0000 7f99c5508640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99a406bc70 0x7f99a406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:13.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.515+0000 7f99c5508640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99a406bc70 0x7f99a406e130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f99bc003aa0 tx=0x7f99bc009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:13.512 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.519+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99b8054ce0 con 0x7f99c015cc30 2024-02-18T11:48:13.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.727+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99c0074520 con 0x7f99c015cc30 2024-02-18T11:48:13.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.728+0000 7f99b5ffb640 1 -- 172.21.15.116:0/467703979 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f99b80583a0 con 0x7f99c015cc30 2024-02-18T11:48:13.721 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-18T11:48:13.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99a406bc70 msgr2=0x7f99a406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99a406bc70 0x7f99a406e130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f99bc003aa0 tx=0x7f99bc009830 comp rx=0 tx=0).stop 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 msgr2=0x7f99c0151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 0x7f99c0151f30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f99b800c410 tx=0x7f99b800c8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 shutdown_connections 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99a406bc70 0x7f99a406e130 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c015cc30 0x7f99c0151f30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99c00d0730 0x7f99c01519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 --2- 172.21.15.116:0/467703979 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99c00b0750 0x7f99c0141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:13.724 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 >> 172.21.15.116:0/467703979 conn(0x7f99c00b3380 msgr2=0x7f99c006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:13.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 shutdown_connections 2024-02-18T11:48:13.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:13.730+0000 7f99c776c640 1 -- 172.21.15.116:0/467703979 wait complete. 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-18T11:48:13.734 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:48:13.735 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-02-18T11:48:13.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.735 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:13.752 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42738sec preferred_lft 42738sec 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:13.753 INFO:teuthology.orchestra.run.smithi143.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 76:d6:de:39:34:37 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::505a:ecff:fe76:7851/64 scope link tentative 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 76:d6:de:39:34:37 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::74d6:deff:fe39:3437/64 scope link tentative 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:13.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.755 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:48:13.818 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-02-18T11:48:13.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.820 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-02-18T11:48:13.877 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-02-18T11:48:13.878 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:48:13.878 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-18T11:48:13.878 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-18T11:48:13.878 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:48:13.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.878 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:13.878 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:13.878 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-18T11:48:13.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-18T11:48:13.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-18T11:48:13.879 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:13.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.973 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-02-18T11:48:13.988 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-02-18T11:48:13.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:13.989 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-02-18T11:48:14.043 INFO:teuthology.orchestra.run.smithi143.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-02-18T11:48:14.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.043 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:14.044 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:14.044 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-02-18T11:48:14.044 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-02-18T11:48:14.044 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:14.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.126 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.193 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:14.194 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether a6:33:30:5f:09:ad brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-18T11:48:14.194 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-18T11:48:14.194 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.194 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::a433:30ff:fe5f:9ad/64 scope link tentative 2024-02-18T11:48:14.194 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.195 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether a6:33:30:5f:09:ad brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-02-18T11:48:14.262 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-02-18T11:48:14.263 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.263 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::a433:30ff:fe5f:9ad/64 scope link tentative 2024-02-18T11:48:14.263 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:14.264 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-02-18T11:48:14.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-02-18T11:48:14.264 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:14.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.390 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-02-18T11:48:14.390 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-02-18T11:48:14.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.1 up 2024-02-18T11:48:14.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.1 master ceph-brx 2024-02-18T11:48:14.390 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-02-18T11:48:14.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:14.432 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-02-18T11:48:14.486 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-02-18T11:48:14.486 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:14.486 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-02-18T11:48:14.541 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-18T11:48:14.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-02-18T11:48:14.626 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-02-18T11:48:14.626 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-02-18T11:48:14.626 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-02-18T11:48:14.634 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-02-18T11:48:14.634 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-02-18T11:48:14.634 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-02-18T11:48:14.635 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:14.635 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:14.644 INFO:teuthology.orchestra.run.smithi143.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:invalid new device string format 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: resolved to: "172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790" 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790:/ 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413" will pass to kernel 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout: You just mounted a file system that supports labels which does not 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout: applications will generate AVC messages and not be allowed access to 2024-02-18T11:48:14.645 INFO:teuthology.orchestra.run.smithi143.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-18T11:48:14.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-18T11:48:14.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-02-18T11:48:14.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.717 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:14.772 INFO:teuthology.orchestra.run.smithi143.stdout:client4905 2024-02-18T11:48:14.772 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:14.772 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:14.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.828 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:14.884 INFO:teuthology.orchestra.run.smithi143.stdout:client4905 2024-02-18T11:48:14.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.885 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:14.940 INFO:teuthology.orchestra.run.smithi143.stdout:client4905 2024-02-18T11:48:14.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:14.941 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/82e3e917-bb19-48fa-a66a-62df77a0f413.client4905/status') 2024-02-18T11:48:15.007 INFO:teuthology.orchestra.run.smithi143.stdout:instance: client.4905 (3)172.21.15.143:0/3298396519 2024-02-18T11:48:15.007 INFO:teuthology.orchestra.run.smithi143.stdout:blocklisted: false 2024-02-18T11:48:15.007 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records in 2024-02-18T11:48:15.007 INFO:teuthology.orchestra.run.smithi143.stderr:0+1 records out 2024-02-18T11:48:15.007 INFO:teuthology.orchestra.run.smithi143.stderr:71 bytes copied, 3.652e-05 s, 1.9 MB/s 2024-02-18T11:48:15.008 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-02-18T11:48:15.024 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.143+0000 7f89ed04e640 1 Processor -- start 2024-02-18T11:48:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.143+0000 7f89ed04e640 1 -- start start 2024-02-18T11:48:15.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89ed04e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f89e8155e30 0x7f89e815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89ed04e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f89e815bdc0 0x7f89e815c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89ed04e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e806d6f0 con 0x7f89e80b1740 2024-02-18T11:48:15.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89ed04e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e815c880 con 0x7f89e8155e30 2024-02-18T11:48:15.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89ed04e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e815cb60 con 0x7f89e815bdc0 2024-02-18T11:48:15.138 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f89e8155e30 0x7f89e815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f89e8155e30 0x7f89e815cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39588/0 (socket says 172.21.15.116:39588) 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 -- 172.21.15.116:0/2915564880 learned_addr learned my addr 172.21.15.116:0/2915564880 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e659c640 1 --2- 172.21.15.116:0/2915564880 >> v2:172.21.15.143:3300/0 conn(0x7f89e815bdc0 0x7f89e815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 -- 172.21.15.116:0/2915564880 >> v2:172.21.15.143:3300/0 conn(0x7f89e815bdc0 msgr2=0x7f89e815c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 --2- 172.21.15.116:0/2915564880 >> v2:172.21.15.143:3300/0 conn(0x7f89e815bdc0 0x7f89e815c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 -- 172.21.15.116:0/2915564880 >> v1:172.21.15.143:6789/0 conn(0x7f89e80b1740 legacy=0x7f89e806db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e6d9d640 1 -- 172.21.15.116:0/2915564880 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e806d380 con 0x7f89e8155e30 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.144+0000 7f89e659c640 1 --2- 172.21.15.116:0/2915564880 >> v2:172.21.15.143:3300/0 conn(0x7f89e815bdc0 0x7f89e815c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.145+0000 7f89e6d9d640 1 --2- 172.21.15.116:0/2915564880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e815cf80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f89e80b4e10 tx=0x7f89d802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac5fc6c2d67f0fc server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.145+0000 7f89e5d9b640 1 -- 172.21.15.116:0/2915564880 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8030860 con 0x7f89e8155e30 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.145+0000 7f89e5d9b640 1 -- 172.21.15.116:0/2915564880 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d80309c0 con 0x7f89e8155e30 2024-02-18T11:48:15.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.145+0000 7f89e5d9b640 1 -- 172.21.15.116:0/2915564880 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8030d50 con 0x7f89e8155e30 2024-02-18T11:48:15.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.145+0000 7f89ed04e640 1 -- 172.21.15.116:0/2915564880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 msgr2=0x7f89e815cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.145+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2915564880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e815cf80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f89e80b4e10 tx=0x7f89d802b9a0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.146+0000 7f89ed04e640 1 -- 172.21.15.116:0/2915564880 shutdown_connections 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.146+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2915564880 >> v2:172.21.15.143:3300/0 conn(0x7f89e815bdc0 0x7f89e815c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.146+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2915564880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e815cf80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.146+0000 7f89ed04e640 1 -- 172.21.15.116:0/2915564880 >> 172.21.15.116:0/2915564880 conn(0x7f89e8066f20 msgr2=0x7f89e806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.146+0000 7f89ed04e640 1 -- 172.21.15.116:0/2915564880 shutdown_connections 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.146+0000 7f89ed04e640 1 -- 172.21.15.116:0/2915564880 wait complete. 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.147+0000 7f89ed04e640 1 Processor -- start 2024-02-18T11:48:15.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.147+0000 7f89ed04e640 1 -- start start 2024-02-18T11:48:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89ed04e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89e80b1740 0x7f89e814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89ed04e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e80cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89ed04e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e815bdc0 0x7f89e814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89ed04e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80cd350 con 0x7f89e8155e30 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89ed04e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e814d450 con 0x7f89e815bdc0 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89ed04e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8150a90 con 0x7f89e80b1740 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e80cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e80cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39598/0 (socket says 172.21.15.116:39598) 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 -- 172.21.15.116:0/2049663358 learned_addr learned my addr 172.21.15.116:0/2049663358 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e759e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e815bdc0 0x7f89e814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e6d9d640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89e80b1740 0x7f89e814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 -- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89e80b1740 msgr2=0x7f89e814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89e80b1740 0x7f89e814d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 -- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e815bdc0 msgr2=0x7f89e814cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.148+0000 7f89e659c640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e815bdc0 0x7f89e814cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89e659c640 1 -- 172.21.15.116:0/2049663358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d80094d0 con 0x7f89e8155e30 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89e659c640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e80cce10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d7d0 tx=0x7f89dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc004030 con 0x7f89e8155e30 2024-02-18T11:48:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc004190 con 0x7f89e8155e30 2024-02-18T11:48:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e8150d70 con 0x7f89e8155e30 2024-02-18T11:48:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc004520 con 0x7f89e8155e30 2024-02-18T11:48:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.149+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e80af5d0 con 0x7f89e8155e30 2024-02-18T11:48:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.150+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e8074520 con 0x7f89e8155e30 2024-02-18T11:48:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.150+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f89dc0096a0 con 0x7f89e8155e30 2024-02-18T11:48:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.151+0000 7f89cf7fe640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f89d406bc70 0x7f89d406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.151+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f89dc08ab70 con 0x7f89e8155e30 2024-02-18T11:48:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.151+0000 7f89e6d9d640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f89d406bc70 0x7f89d406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.151+0000 7f89e6d9d640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f89d406bc70 0x7f89d406e130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89d8036000 tx=0x7f89d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.155+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89dc055070 con 0x7f89e8155e30 2024-02-18T11:48:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.327+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e80cd4f0 con 0x7f89e8155e30 2024-02-18T11:48:15.321 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.328+0000 7f89cf7fe640 1 -- 172.21.15.116:0/2049663358 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f89dc058730 con 0x7f89e8155e30 2024-02-18T11:48:15.322 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:15.322 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":25,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:14.683533+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f89d406bc70 msgr2=0x7f89d406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f89d406bc70 0x7f89d406e130 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89d8036000 tx=0x7f89d8008030 comp rx=0 tx=0).stop 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 msgr2=0x7f89e80cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e80cce10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d7d0 tx=0x7f89dc00dc90 comp rx=0 tx=0).stop 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 shutdown_connections 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f89d406bc70 0x7f89d406e130 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89e815bdc0 0x7f89e814cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89e8155e30 0x7f89e80cce10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 --2- 172.21.15.116:0/2049663358 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89e80b1740 0x7f89e814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 >> 172.21.15.116:0/2049663358 conn(0x7f89e8066f20 msgr2=0x7f89e80b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 shutdown_connections 2024-02-18T11:48:15.324 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.330+0000 7f89ed04e640 1 -- 172.21.15.116:0/2049663358 wait complete. 2024-02-18T11:48:15.335 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:15.335 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:15.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.445+0000 7fa9d80d4640 1 Processor -- start 2024-02-18T11:48:15.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.445+0000 7fa9d80d4640 1 -- start start 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d80d4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d015dba0 0x7fa9d0161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d80d4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa9d00b93b0 0x7fa9d00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d80d4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d00cf180 con 0x7fa9d00ced30 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d80d4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d00d21b0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d80d4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00d1e60 con 0x7fa9d015dba0 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d566f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa9d00b93b0 0x7fa9d00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d566f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa9d00b93b0 0x7fa9d00ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39604/0 (socket says 172.21.15.116:39604) 2024-02-18T11:48:15.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.446+0000 7fa9d566f640 1 -- 172.21.15.116:0/3767719275 learned_addr learned my addr 172.21.15.116:0/3767719275 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d5e70640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d015dba0 0x7fa9d0161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d566f640 1 -- 172.21.15.116:0/3767719275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d015dba0 msgr2=0x7fa9d0161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d566f640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d015dba0 0x7fa9d0161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d566f640 1 -- 172.21.15.116:0/3767719275 >> v1:172.21.15.143:6789/0 conn(0x7fa9d00ced30 legacy=0x7fa9d006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d566f640 1 -- 172.21.15.116:0/3767719275 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0162050 con 0x7fa9d00b93b0 2024-02-18T11:48:15.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d5e70640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d015dba0 0x7fa9d0161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:15.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d566f640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d00ce7f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009870 tx=0x7fa9c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14c94b2b03dba46 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d4e6e640 1 -- 172.21.15.116:0/3767719275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0030840 con 0x7fa9d00b93b0 2024-02-18T11:48:15.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d4e6e640 1 -- 172.21.15.116:0/3767719275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c00309a0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.447+0000 7fa9d4e6e640 1 -- 172.21.15.116:0/3767719275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0030d30 con 0x7fa9d00b93b0 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3767719275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 msgr2=0x7fa9d00ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d00ce7f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009870 tx=0x7fa9c002b980 comp rx=0 tx=0).stop 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3767719275 shutdown_connections 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d00ce7f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3767719275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d015dba0 0x7fa9d0161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3767719275 >> 172.21.15.116:0/3767719275 conn(0x7fa9d00d04a0 msgr2=0x7fa9d00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3767719275 shutdown_connections 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.448+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3767719275 wait complete. 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.449+0000 7fa9d80d4640 1 Processor -- start 2024-02-18T11:48:15.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.449+0000 7fa9d80d4640 1 -- start start 2024-02-18T11:48:15.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d80d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d80d4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d00ced30 0x7fa9d00c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d80d4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9d015dba0 0x7fa9d00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d80d4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00cab90 con 0x7fa9d00b93b0 2024-02-18T11:48:15.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d80d4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00cd1b0 con 0x7fa9d015dba0 2024-02-18T11:48:15.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d80d4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00cae30 con 0x7fa9d00ced30 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39612/0 (socket says 172.21.15.116:39612) 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 -- 172.21.15.116:0/3588309833 learned_addr learned my addr 172.21.15.116:0/3588309833 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d6671640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9d015dba0 0x7fa9d00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d566f640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d00ced30 0x7fa9d00c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 -- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d00ced30 msgr2=0x7fa9d00c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d00ced30 0x7fa9d00c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 -- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9d015dba0 msgr2=0x7fa9d00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9d015dba0 0x7fa9d00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 -- 172.21.15.116:0/3588309833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c00094d0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d566f640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d00ced30 0x7fa9d00c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.450+0000 7fa9d5e70640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d015b730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00ce00 tx=0x7fa9cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.451+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00f890 con 0x7fa9d00b93b0 2024-02-18T11:48:15.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.451+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00f9f0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.451+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d00cb110 con 0x7fa9d00b93b0 2024-02-18T11:48:15.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.451+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00fdc0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.451+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d00cd490 con 0x7fa9d00b93b0 2024-02-18T11:48:15.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.452+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d00cb2a0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.452+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa9cc00b040 con 0x7fa9d00b93b0 2024-02-18T11:48:15.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.452+0000 7fa9c67fc640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa9b406bc70 0x7fa9b406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.452+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fa9cc08b570 con 0x7fa9d00b93b0 2024-02-18T11:48:15.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.453+0000 7fa9d566f640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa9b406bc70 0x7fa9b406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.453+0000 7fa9d566f640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa9b406bc70 0x7fa9b406e130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0036000 tx=0x7fa9c0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.457+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9cc055930 con 0x7fa9d00b93b0 2024-02-18T11:48:15.621 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.628+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9d00d1730 con 0x7fa9d00b93b0 2024-02-18T11:48:15.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.628+0000 7fa9c67fc640 1 -- 172.21.15.116:0/3588309833 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9cc058ff0 con 0x7fa9d00b93b0 2024-02-18T11:48:15.622 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa9b406bc70 msgr2=0x7fa9b406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa9b406bc70 0x7fa9b406e130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0036000 tx=0x7fa9c0008030 comp rx=0 tx=0).stop 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 msgr2=0x7fa9d015b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d015b730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00ce00 tx=0x7fa9cc00a650 comp rx=0 tx=0).stop 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 shutdown_connections 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa9b406bc70 0x7fa9b406e130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9d015dba0 0x7fa9d00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa9d00ced30 0x7fa9d00c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.624 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 --2- 172.21.15.116:0/3588309833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9d00b93b0 0x7fa9d015b730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 >> 172.21.15.116:0/3588309833 conn(0x7fa9d00d04a0 msgr2=0x7fa9d00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:15.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 shutdown_connections 2024-02-18T11:48:15.625 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.630+0000 7fa9d80d4640 1 -- 172.21.15.116:0/3588309833 wait complete. 2024-02-18T11:48:15.635 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:15.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.750+0000 7fbbcf0f2640 1 Processor -- start 2024-02-18T11:48:15.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.750+0000 7fbbcf0f2640 1 -- start start 2024-02-18T11:48:15.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbcf0f2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbbc80b31a0 0x7fbbc806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbcf0f2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80b3610 con 0x7fbbc8144090 2024-02-18T11:48:15.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbcf0f2640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc806eca0 con 0x7fbbc806e870 2024-02-18T11:48:15.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbcf0f2640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc80b2a50 con 0x7fbbc80b31a0 2024-02-18T11:48:15.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbcce8e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbbc8144090 0x7fbbc80cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59740/0 (socket says 172.21.15.116:59740) 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbcce8e640 1 -- 172.21.15.116:0/153812499 learned_addr learned my addr 172.21.15.116:0/153812499 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1305376283 0 0) 0x7fbbc80b3610 con 0x7fbbc8144090 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.751+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb8003520 con 0x7fbbc8144090 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc7fff640 1 --2- 172.21.15.116:0/153812499 >> v2:172.21.15.143:3300/0 conn(0x7fbbc80b31a0 0x7fbbc806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2294837113 0 0) 0x7fbbb8003520 con 0x7fbbc8144090 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 >> v2:172.21.15.143:3300/0 conn(0x7fbbc80b31a0 msgr2=0x7fbbc806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 --2- 172.21.15.116:0/153812499 >> v2:172.21.15.143:3300/0 conn(0x7fbbc80b31a0 0x7fbbc806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 >> v1:172.21.15.143:6789/0 conn(0x7fbbc806e870 legacy=0x7fbbc815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80cd0f0 con 0x7fbbc8144090 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fbbbc001670 con 0x7fbbc8144090 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 >> v1:172.21.15.116:6789/0 conn(0x7fbbc8144090 legacy=0x7fbbc80cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb8003970 0x7fbbb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.748 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80047c0 0x7fbbb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbb80052a0 0x7fbbb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80b2a50 con 0x7fbbb80052a0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb800c170 con 0x7fbbb8003970 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb800c330 con 0x7fbbb80047c0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcd68f640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbb80052a0 0x7fbbb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcce8e640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb8003970 0x7fbbb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbc7fff640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80047c0 0x7fbbb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcd68f640 1 -- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80047c0 msgr2=0x7fbbb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcd68f640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80047c0 0x7fbbb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcd68f640 1 -- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb8003970 msgr2=0x7fbbb8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcd68f640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbb8003970 0x7fbbb8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.752+0000 7fbbcd68f640 1 -- 172.21.15.116:0/153812499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8003520 con 0x7fbbb80052a0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbcd68f640 1 -- 172.21.15.116:0/153812499 reap_dead start 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbcd68f640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbb80052a0 0x7fbbb800b980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0001e20 tx=0x7fbbb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fd212901255533 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0003b70 con 0x7fbbb80052a0 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc0003cd0 con 0x7fbbb80052a0 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbc77fe640 1 -- 172.21.15.116:0/153812499 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0005310 con 0x7fbbb80052a0 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/153812499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbb80052a0 msgr2=0x7fbbb800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.753+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbb80052a0 0x7fbbb800b980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0001e20 tx=0x7fbbb80057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.754+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/153812499 shutdown_connections 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.754+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/153812499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbb80052a0 0x7fbbb800b980 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.754+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/153812499 >> 172.21.15.116:0/153812499 conn(0x7fbbc8067000 msgr2=0x7fbbc806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.754+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/153812499 shutdown_connections 2024-02-18T11:48:15.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.754+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/153812499 wait complete. 2024-02-18T11:48:15.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.755+0000 7fbbcf0f2640 1 Processor -- start 2024-02-18T11:48:15.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.755+0000 7fbbcf0f2640 1 -- start start 2024-02-18T11:48:15.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80052a0 0x7fbbc80ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 0x7fbbc815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbc813f7d0 0x7fbbc813fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80ace00 con 0x7fbbc813eea0 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80ad080 con 0x7fbbc813f7d0 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80ad360 con 0x7fbbb80052a0 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 0x7fbbc815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 0x7fbbc815d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39630/0 (socket says 172.21.15.116:39630) 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 -- 172.21.15.116:0/2761757858 learned_addr learned my addr 172.21.15.116:0/2761757858 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcd68f640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbc813f7d0 0x7fbbc813fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 -- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80052a0 msgr2=0x7fbbc80ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80052a0 0x7fbbc80ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 -- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbc813f7d0 msgr2=0x7fbbc813fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.755 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbc813f7d0 0x7fbbc813fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 -- 172.21.15.116:0/2761757858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80ad640 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcd68f640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbc813f7d0 0x7fbbc813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc7fff640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 0x7fbbc815d240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400b7c0 tx=0x7fbbb400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4010830 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4010990 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc80ad920 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4010d20 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.756+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc80ae120 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.757+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc80cbe40 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.758+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fbbb4014430 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.758+0000 7fbbc57fa640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbbac06bc70 0x7fbbac06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.758+0000 7fbbcce8e640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbbac06bc70 0x7fbbac06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.758+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fbbb408b160 con 0x7fbbc813eea0 2024-02-18T11:48:15.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.759+0000 7fbbcce8e640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbbac06bc70 0x7fbbac06e130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001050 tx=0x7fbbbc000fe0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:15.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.762+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbbb40555b0 con 0x7fbbc813eea0 2024-02-18T11:48:15.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.933+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbc8074520 con 0x7fbbc813eea0 2024-02-18T11:48:15.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.934+0000 7fbbc57fa640 1 -- 172.21.15.116:0/2761757858 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbb4058c70 con 0x7fbbc813eea0 2024-02-18T11:48:15.927 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbbac06bc70 msgr2=0x7fbbac06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbbac06bc70 0x7fbbac06e130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc001050 tx=0x7fbbbc000fe0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 msgr2=0x7fbbc815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 0x7fbbc815d240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbbb400b7c0 tx=0x7fbbb400bc80 comp rx=0 tx=0).stop 2024-02-18T11:48:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 shutdown_connections 2024-02-18T11:48:15.929 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbbac06bc70 0x7fbbac06e130 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbbc813f7d0 0x7fbbc813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbbc813eea0 0x7fbbc815d240 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 --2- 172.21.15.116:0/2761757858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbbb80052a0 0x7fbbc80ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 >> 172.21.15.116:0/2761757858 conn(0x7fbbc8067000 msgr2=0x7fbbc813dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 shutdown_connections 2024-02-18T11:48:15.930 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:15.936+0000 7fbbcf0f2640 1 -- 172.21.15.116:0/2761757858 wait complete. 2024-02-18T11:48:15.940 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:16.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.049+0000 7f0f1065b640 1 Processor -- start 2024-02-18T11:48:16.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.050+0000 7f0f1065b640 1 -- start start 2024-02-18T11:48:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.050+0000 7f0f1065b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0f0815b9b0 0x7f0f0815fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.050+0000 7f0f1065b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f080cea10 0x7f0f080cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.050+0000 7f0f1065b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f080b2a60 con 0x7f0f080cc640 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f1065b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f08160300 con 0x7f0f080cea10 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f1065b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0806fb90 con 0x7f0f0815b9b0 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f080cea10 0x7f0f080cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f080cea10 0x7f0f080cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39642/0 (socket says 172.21.15.116:39642) 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0ebf8640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0f080cc640 0x7f0f080ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59752/0 (socket says 172.21.15.116:59752) 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 -- 172.21.15.116:0/3447276121 learned_addr learned my addr 172.21.15.116:0/3447276121 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0e3f7640 1 --2- 172.21.15.116:0/3447276121 >> v2:172.21.15.143:3300/0 conn(0x7f0f0815b9b0 0x7f0f0815fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 -- 172.21.15.116:0/3447276121 >> v2:172.21.15.143:3300/0 conn(0x7f0f0815b9b0 msgr2=0x7f0f0815fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 --2- 172.21.15.116:0/3447276121 >> v2:172.21.15.143:3300/0 conn(0x7f0f0815b9b0 0x7f0f0815fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 -- 172.21.15.116:0/3447276121 >> v1:172.21.15.116:6789/0 conn(0x7f0f080cc640 legacy=0x7f0f080ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0dbf6640 1 -- 172.21.15.116:0/3447276121 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f080b2d00 con 0x7f0f080cea10 2024-02-18T11:48:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.051+0000 7f0f0d3f5640 1 -- 172.21.15.116:0/3447276121 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846854277 0 0) 0x7f0f080b2a60 con 0x7f0f080cc640 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f0dbf6640 1 --2- 172.21.15.116:0/3447276121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f080cea10 0x7f0f080cee70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f08072dc0 tx=0x7f0ef802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3903d1ab6f661bdb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f0d3f5640 1 -- 172.21.15.116:0/3447276121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8030860 con 0x7f0f080cea10 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f0d3f5640 1 -- 172.21.15.116:0/3447276121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef80309c0 con 0x7f0f080cea10 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f0d3f5640 1 -- 172.21.15.116:0/3447276121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8030d50 con 0x7f0f080cea10 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f1065b640 1 -- 172.21.15.116:0/3447276121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f080cea10 msgr2=0x7f0f080cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f1065b640 1 --2- 172.21.15.116:0/3447276121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f080cea10 0x7f0f080cee70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f08072dc0 tx=0x7f0ef802b9a0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f1065b640 1 -- 172.21.15.116:0/3447276121 shutdown_connections 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f1065b640 1 --2- 172.21.15.116:0/3447276121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f080cea10 0x7f0f080cee70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.052+0000 7f0f1065b640 1 --2- 172.21.15.116:0/3447276121 >> v2:172.21.15.143:3300/0 conn(0x7f0f0815b9b0 0x7f0f0815fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.053+0000 7f0f1065b640 1 -- 172.21.15.116:0/3447276121 >> 172.21.15.116:0/3447276121 conn(0x7f0f080cf9b0 msgr2=0x7f0f080cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.053+0000 7f0f1065b640 1 -- 172.21.15.116:0/3447276121 shutdown_connections 2024-02-18T11:48:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.053+0000 7f0f1065b640 1 -- 172.21.15.116:0/3447276121 wait complete. 2024-02-18T11:48:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 Processor -- start 2024-02-18T11:48:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 -- start start 2024-02-18T11:48:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f080cc640 0x7f0f080b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f080cea10 0x7f0f080aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 0x7f0f080aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0815cde0 con 0x7f0f0815b9b0 2024-02-18T11:48:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0815d030 con 0x7f0f080cea10 2024-02-18T11:48:16.050 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f1065b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0815d310 con 0x7f0f080cc640 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f0ebf8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 0x7f0f080aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f0ebf8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 0x7f0f080aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39656/0 (socket says 172.21.15.116:39656) 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.054+0000 7f0f0ebf8640 1 -- 172.21.15.116:0/2177042035 learned_addr learned my addr 172.21.15.116:0/2177042035 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0dbf6640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f080cea10 0x7f0f080aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0ebf8640 1 -- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f080cc640 msgr2=0x7f0f080b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0ebf8640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f080cc640 0x7f0f080b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0ebf8640 1 -- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f080cea10 msgr2=0x7f0f080aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0ebf8640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f080cea10 0x7f0f080aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0ebf8640 1 -- 172.21.15.116:0/2177042035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef80094d0 con 0x7f0f0815b9b0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f0ebf8640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 0x7f0f080aaa20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c780 tx=0x7f0f0000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00004d60 con 0x7f0f0815b9b0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0000a640 con 0x7f0f0815b9b0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0815d5f0 con 0x7f0f0815b9b0 2024-02-18T11:48:16.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000aa10 con 0x7f0f0815b9b0 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.055+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0815dbb0 con 0x7f0f0815b9b0 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.056+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f08067840 con 0x7f0f0815b9b0 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.056+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f0f0000ab70 con 0x7f0f0815b9b0 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.057+0000 7f0efeffd640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0eec06bc70 0x7f0eec06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.057+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f0f0008a250 con 0x7f0f0815b9b0 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.057+0000 7f0f0e3f7640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0eec06bc70 0x7f0eec06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.057+0000 7f0f0e3f7640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0eec06bc70 0x7f0eec06e130 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.061+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f00054610 con 0x7f0f0815b9b0 2024-02-18T11:48:16.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.232+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f080d0690 con 0x7f0f0815b9b0 2024-02-18T11:48:16.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.233+0000 7f0efeffd640 1 -- 172.21.15.116:0/2177042035 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f00057cd0 con 0x7f0f0815b9b0 2024-02-18T11:48:16.227 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:16.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0eec06bc70 msgr2=0x7f0eec06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0eec06bc70 0x7f0eec06e130 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006fd0 tx=0x7f0f04008040 comp rx=0 tx=0).stop 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 msgr2=0x7f0f080aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 0x7f0f080aaa20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c780 tx=0x7f0f0000cb80 comp rx=0 tx=0).stop 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 shutdown_connections 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0eec06bc70 0x7f0eec06e130 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f0815b9b0 0x7f0f080aaa20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f080cea10 0x7f0f080aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 --2- 172.21.15.116:0/2177042035 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f080cc640 0x7f0f080b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 >> 172.21.15.116:0/2177042035 conn(0x7f0f080cf9b0 msgr2=0x7f0f08157690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 shutdown_connections 2024-02-18T11:48:16.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.235+0000 7f0f1065b640 1 -- 172.21.15.116:0/2177042035 wait complete. 2024-02-18T11:48:16.240 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:16.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.350+0000 7f2d2834f640 1 Processor -- start 2024-02-18T11:48:16.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.350+0000 7f2d2834f640 1 -- start start 2024-02-18T11:48:16.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d2834f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d20144090 0x7f2d200b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d2834f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d2834f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d200b2a50 con 0x7f2d200cce20 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d2834f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2015b090 con 0x7f2d20144090 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d2834f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2015b370 con 0x7f2d200cc9e0 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d20144090 0x7f2d200b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d20144090 0x7f2d200b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39666/0 (socket says 172.21.15.116:39666) 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 -- 172.21.15.116:0/3733380459 learned_addr learned my addr 172.21.15.116:0/3733380459 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d268ec640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2d200cce20 0x7f2d2015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59768/0 (socket says 172.21.15.116:59768) 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d258ea640 1 --2- 172.21.15.116:0/3733380459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 -- 172.21.15.116:0/3733380459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 msgr2=0x7f2d2006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 --2- 172.21.15.116:0/3733380459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 -- 172.21.15.116:0/3733380459 >> v1:172.21.15.116:6789/0 conn(0x7f2d200cce20 legacy=0x7f2d2015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d260eb640 1 -- 172.21.15.116:0/3733380459 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2015b650 con 0x7f2d20144090 2024-02-18T11:48:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.351+0000 7f2d250e9640 1 -- 172.21.15.116:0/3733380459 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780685019 0 0) 0x7f2d200b2a50 con 0x7f2d200cce20 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.352+0000 7f2d260eb640 1 --2- 172.21.15.116:0/3733380459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d20144090 0x7f2d200b31a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009e50 tx=0x7f2d1802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ce0e9c610baa46 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.352+0000 7f2d250e9640 1 -- 172.21.15.116:0/3733380459 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1802ca70 con 0x7f2d20144090 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.352+0000 7f2d250e9640 1 -- 172.21.15.116:0/3733380459 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1802cbd0 con 0x7f2d20144090 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.352+0000 7f2d250e9640 1 -- 172.21.15.116:0/3733380459 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180343f0 con 0x7f2d20144090 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 -- 172.21.15.116:0/3733380459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d20144090 msgr2=0x7f2d200b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3733380459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d20144090 0x7f2d200b31a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009e50 tx=0x7f2d1802bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 -- 172.21.15.116:0/3733380459 shutdown_connections 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3733380459 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3733380459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d20144090 0x7f2d200b31a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 -- 172.21.15.116:0/3733380459 >> 172.21.15.116:0/3733380459 conn(0x7f2d20067000 msgr2=0x7f2d2006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 -- 172.21.15.116:0/3733380459 shutdown_connections 2024-02-18T11:48:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.353+0000 7f2d2834f640 1 -- 172.21.15.116:0/3733380459 wait complete. 2024-02-18T11:48:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.354+0000 7f2d2834f640 1 Processor -- start 2024-02-18T11:48:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.354+0000 7f2d2834f640 1 -- start start 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.354+0000 7f2d2834f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 0x7f2d200b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d20144090 0x7f2d2012e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2012f670 con 0x7f2d200cce20 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2015dd70 con 0x7f2d20144090 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2012f0b0 con 0x7f2d200cc9e0 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 0x7f2d200b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 0x7f2d200b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39668/0 (socket says 172.21.15.116:39668) 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 -- 172.21.15.116:0/3338249299 learned_addr learned my addr 172.21.15.116:0/3338249299 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d268ec640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d20144090 0x7f2d2012e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d260eb640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 -- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 msgr2=0x7f2d2015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 -- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d20144090 msgr2=0x7f2d2012e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d20144090 0x7f2d2012e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 -- 172.21.15.116:0/3338249299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009b30 con 0x7f2d200cce20 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d268ec640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d20144090 0x7f2d2012e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d258ea640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 0x7f2d200b3b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000d5f0 tx=0x7f2d1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d100125f0 con 0x7f2d200cce20 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d10012750 con 0x7f2d200cce20 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2012f390 con 0x7f2d200cce20 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10012b00 con 0x7f2d200cce20 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.355+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d200b3620 con 0x7f2d200cce20 2024-02-18T11:48:16.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.356+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d200ce9f0 con 0x7f2d200cce20 2024-02-18T11:48:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.357+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f2d10012c60 con 0x7f2d200cce20 2024-02-18T11:48:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.357+0000 7f2d16ffd640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2d0406bc70 0x7f2d0406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.357+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2d1008b060 con 0x7f2d200cce20 2024-02-18T11:48:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.357+0000 7f2d260eb640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2d0406bc70 0x7f2d0406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.357+0000 7f2d260eb640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2d0406bc70 0x7f2d0406e130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d18035000 tx=0x7f2d18030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.361+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d10055420 con 0x7f2d200cce20 2024-02-18T11:48:16.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.532+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2d200cbe40 con 0x7f2d200cce20 2024-02-18T11:48:16.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.532+0000 7f2d16ffd640 1 -- 172.21.15.116:0/3338249299 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2d10058ae0 con 0x7f2d200cce20 2024-02-18T11:48:16.526 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2d0406bc70 msgr2=0x7f2d0406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2d0406bc70 0x7f2d0406e130 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d18035000 tx=0x7f2d18030000 comp rx=0 tx=0).stop 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 msgr2=0x7f2d200b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 0x7f2d200b3b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000d5f0 tx=0x7f2d1000dab0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 shutdown_connections 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2d0406bc70 0x7f2d0406e130 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d20144090 0x7f2d2012e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d200cce20 0x7f2d200b3b40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 --2- 172.21.15.116:0/3338249299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2d200cc9e0 0x7f2d2015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 >> 172.21.15.116:0/3338249299 conn(0x7f2d20067000 msgr2=0x7f2d200aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.528 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 shutdown_connections 2024-02-18T11:48:16.529 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.534+0000 7f2d2834f640 1 -- 172.21.15.116:0/3338249299 wait complete. 2024-02-18T11:48:16.539 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.645+0000 7ff5cf325640 1 Processor -- start 2024-02-18T11:48:16.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.645+0000 7ff5cf325640 1 -- start start 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cf325640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5c80afe40 0x7ff5c806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cf325640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5c806b0a0 0x7ff5c80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cf325640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c81625d0 con 0x7ff5c815daa0 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cf325640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c8073100 con 0x7ff5c80afe40 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cf325640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c8161810 con 0x7ff5c806b0a0 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cc8c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5c80afe40 0x7ff5c806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cc8c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5c80afe40 0x7ff5c806ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39670/0 (socket says 172.21.15.116:39670) 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.646+0000 7ff5cc8c0640 1 -- 172.21.15.116:0/3327242930 learned_addr learned my addr 172.21.15.116:0/3327242930 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.640 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.647+0000 7ff5cc8c0640 1 -- 172.21.15.116:0/3327242930 >> v2:172.21.15.143:3300/0 conn(0x7ff5c806b0a0 msgr2=0x7ff5c80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.647+0000 7ff5cc8c0640 1 --2- 172.21.15.116:0/3327242930 >> v2:172.21.15.143:3300/0 conn(0x7ff5c806b0a0 0x7ff5c80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.647+0000 7ff5cc8c0640 1 -- 172.21.15.116:0/3327242930 >> v1:172.21.15.143:6789/0 conn(0x7ff5c815daa0 legacy=0x7ff5c80af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:16.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.647+0000 7ff5cc8c0640 1 -- 172.21.15.116:0/3327242930 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8161af0 con 0x7ff5c80afe40 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.647+0000 7ff5cc8c0640 1 --2- 172.21.15.116:0/3327242930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c80afe40 0x7ff5c806ab60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80cd110 tx=0x7ff5b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f88a5d00a7429f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5bffff640 1 -- 172.21.15.116:0/3327242930 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8031870 con 0x7ff5c80afe40 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5bffff640 1 -- 172.21.15.116:0/3327242930 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b80319d0 con 0x7ff5c80afe40 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5bffff640 1 -- 172.21.15.116:0/3327242930 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8031d60 con 0x7ff5c80afe40 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 -- 172.21.15.116:0/3327242930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c80afe40 msgr2=0x7ff5c806ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 --2- 172.21.15.116:0/3327242930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c80afe40 0x7ff5c806ab60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80cd110 tx=0x7ff5b802b910 comp rx=0 tx=0).stop 2024-02-18T11:48:16.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 -- 172.21.15.116:0/3327242930 shutdown_connections 2024-02-18T11:48:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 --2- 172.21.15.116:0/3327242930 >> v2:172.21.15.143:3300/0 conn(0x7ff5c806b0a0 0x7ff5c80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 --2- 172.21.15.116:0/3327242930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c80afe40 0x7ff5c806ab60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 -- 172.21.15.116:0/3327242930 >> 172.21.15.116:0/3327242930 conn(0x7ff5c806e280 msgr2=0x7ff5c806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 -- 172.21.15.116:0/3327242930 shutdown_connections 2024-02-18T11:48:16.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.648+0000 7ff5cf325640 1 -- 172.21.15.116:0/3327242930 wait complete. 2024-02-18T11:48:16.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.649+0000 7ff5cf325640 1 Processor -- start 2024-02-18T11:48:16.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.649+0000 7ff5cf325640 1 -- start start 2024-02-18T11:48:16.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cf325640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 0x7ff5c8148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cf325640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5c80afe40 0x7ff5c8160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cf325640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5c815daa0 0x7ff5c8157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cf325640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81582d0 con 0x7ff5c806b0a0 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cf325640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81428e0 con 0x7ff5c80afe40 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cf325640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8158050 con 0x7ff5c815daa0 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 0x7ff5c8148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 0x7ff5c8148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39674/0 (socket says 172.21.15.116:39674) 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 -- 172.21.15.116:0/2408134694 learned_addr learned my addr 172.21.15.116:0/2408134694 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cc8c0640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5c80afe40 0x7ff5c8160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd8c2640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5c815daa0 0x7ff5c8157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 -- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5c815daa0 msgr2=0x7ff5c8157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5c815daa0 0x7ff5c8157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 -- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5c80afe40 msgr2=0x7ff5c8160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5c80afe40 0x7ff5c8160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.650+0000 7ff5cd0c1640 1 -- 172.21.15.116:0/2408134694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b80094d0 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5cd8c2640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5c815daa0 0x7ff5c8157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5cd0c1640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 0x7ff5c8148d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400ba80 tx=0x7ff5c400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400cc30 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400cd90 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c815a430 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4014680 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.651+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c815a9f0 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.652+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff5c401d3b0 con 0x7ff5c806b0a0 2024-02-18T11:48:16.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.652+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c815a290 con 0x7ff5c806b0a0 2024-02-18T11:48:16.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.653+0000 7ff5bdffb640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5ac06bc70 0x7ff5ac06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.653+0000 7ff5cc8c0640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5ac06bc70 0x7ff5ac06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.653+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7ff5c408aa20 con 0x7ff5c806b0a0 2024-02-18T11:48:16.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.653+0000 7ff5cc8c0640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5ac06bc70 0x7ff5ac06e130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8036000 tx=0x7ff5b803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.657+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5c815a290 con 0x7ff5c806b0a0 2024-02-18T11:48:16.824 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.831+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5c815a290 con 0x7ff5c806b0a0 2024-02-18T11:48:16.825 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.832+0000 7ff5bdffb640 1 -- 172.21.15.116:0/2408134694 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7ff5c815a290 con 0x7ff5c806b0a0 2024-02-18T11:48:16.825 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:16.826 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:16.827 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5ac06bc70 msgr2=0x7ff5ac06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5ac06bc70 0x7ff5ac06e130 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8036000 tx=0x7ff5b803c040 comp rx=0 tx=0).stop 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 msgr2=0x7ff5c8148d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 0x7ff5c8148d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400ba80 tx=0x7ff5c400bf40 comp rx=0 tx=0).stop 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 shutdown_connections 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5ac06bc70 0x7ff5ac06e130 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5c815daa0 0x7ff5c8157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5c80afe40 0x7ff5c8160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 --2- 172.21.15.116:0/2408134694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806b0a0 0x7ff5c8148d20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 >> 172.21.15.116:0/2408134694 conn(0x7ff5c806e280 msgr2=0x7ff5c8070290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 shutdown_connections 2024-02-18T11:48:16.828 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.834+0000 7ff5cf325640 1 -- 172.21.15.116:0/2408134694 wait complete. 2024-02-18T11:48:16.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:16.839 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:16.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.949+0000 7f39e388d640 1 Processor -- start 2024-02-18T11:48:16.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.950+0000 7f39e388d640 1 -- start start 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.950+0000 7f39e388d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f39dc0cee50 0x7f39dc168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.950+0000 7f39e388d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39dc0d1ee0 con 0x7f39dc15dbf0 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.950+0000 7f39e388d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39dc161dc0 con 0x7f39dc1617e0 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.950+0000 7f39e388d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39dc1620a0 con 0x7f39dc0cee50 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1e2a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f39dc0cee50 0x7f39dc168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1e2a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f39dc0cee50 0x7f39dc168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39688/0 (socket says 172.21.15.116:39688) 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1e2a640 1 -- 172.21.15.116:0/2756457504 learned_addr learned my addr 172.21.15.116:0/2756457504 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1629640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f39dc15dbf0 0x7f39dc0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59782/0 (socket says 172.21.15.116:59782) 2024-02-18T11:48:16.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1e2a640 1 -- 172.21.15.116:0/2756457504 >> v1:172.21.15.143:6789/0 conn(0x7f39dc1617e0 legacy=0x7f39dc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:16.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1e2a640 1 -- 172.21.15.116:0/2756457504 >> v1:172.21.15.116:6789/0 conn(0x7f39dc15dbf0 legacy=0x7f39dc0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39e1e2a640 1 -- 172.21.15.116:0/2756457504 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc162380 con 0x7f39dc0cee50 2024-02-18T11:48:16.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.951+0000 7f39d3fff640 1 -- 172.21.15.116:0/2756457504 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326910624 0 0) 0x7f39dc0d1ee0 con 0x7f39dc15dbf0 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e1e2a640 1 --2- 172.21.15.116:0/2756457504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc0cee50 0x7f39dc168b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009e50 tx=0x7f39d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78e2ccc1008415 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39d3fff640 1 -- 172.21.15.116:0/2756457504 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d402ca80 con 0x7f39dc0cee50 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39d3fff640 1 -- 172.21.15.116:0/2756457504 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d402cbe0 con 0x7f39dc0cee50 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39d3fff640 1 -- 172.21.15.116:0/2756457504 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d40343f0 con 0x7f39dc0cee50 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 -- 172.21.15.116:0/2756457504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc0cee50 msgr2=0x7f39dc168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 --2- 172.21.15.116:0/2756457504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc0cee50 0x7f39dc168b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009e50 tx=0x7f39d402bee0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 -- 172.21.15.116:0/2756457504 shutdown_connections 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 --2- 172.21.15.116:0/2756457504 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc0cee50 0x7f39dc168b10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 -- 172.21.15.116:0/2756457504 >> 172.21.15.116:0/2756457504 conn(0x7f39dc0b5480 msgr2=0x7f39dc0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:16.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 -- 172.21.15.116:0/2756457504 shutdown_connections 2024-02-18T11:48:16.947 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.952+0000 7f39e388d640 1 -- 172.21.15.116:0/2756457504 wait complete. 2024-02-18T11:48:16.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.953+0000 7f39e388d640 1 Processor -- start 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 -- start start 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39dc0cee50 0x7f39dc15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39dc15dbf0 0x7f39dc152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 0x7f39dc15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1601f0 con 0x7f39dc1617e0 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc160490 con 0x7f39dc15dbf0 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e388d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc160770 con 0x7f39dc0cee50 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e1e2a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 0x7f39dc15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e1e2a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 0x7f39dc15e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39692/0 (socket says 172.21.15.116:39692) 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.954+0000 7f39e1e2a640 1 -- 172.21.15.116:0/3132351497 learned_addr learned my addr 172.21.15.116:0/3132351497 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e1e2a640 1 -- 172.21.15.116:0/3132351497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39dc0cee50 msgr2=0x7f39dc15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e1e2a640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39dc0cee50 0x7f39dc15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e1e2a640 1 -- 172.21.15.116:0/3132351497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39dc15dbf0 msgr2=0x7f39dc152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e1e2a640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39dc15dbf0 0x7f39dc152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e1e2a640 1 -- 172.21.15.116:0/3132351497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4009b30 con 0x7f39dc1617e0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e1e2a640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 0x7f39dc15e2e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f39d4036040 tx=0x7f39d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d403e450 con 0x7f39dc1617e0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d403e5b0 con 0x7f39dc1617e0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc15e150 con 0x7f39dc1617e0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d403e960 con 0x7f39dc1617e0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.955+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc160d30 con 0x7f39dc1617e0 2024-02-18T11:48:16.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.956+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc067840 con 0x7f39dc1617e0 2024-02-18T11:48:16.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.956+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f39d402cd50 con 0x7f39dc1617e0 2024-02-18T11:48:16.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.957+0000 7f39d1ffb640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f39c006bc70 0x7f39c006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:16.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.957+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f39d40ad7c0 con 0x7f39dc1617e0 2024-02-18T11:48:16.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.957+0000 7f39e1629640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f39c006bc70 0x7f39c006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:16.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.957+0000 7f39e1629640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f39c006bc70 0x7f39c006e130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39cc007fd0 tx=0x7f39cc007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:16.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:16.961+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39d4077ca0 con 0x7f39dc1617e0 2024-02-18T11:48:17.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.131+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39dc074520 con 0x7f39dc1617e0 2024-02-18T11:48:17.126 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.131+0000 7f39d1ffb640 1 -- 172.21.15.116:0/3132351497 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39d407b360 con 0x7f39dc1617e0 2024-02-18T11:48:17.126 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:17.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f39c006bc70 msgr2=0x7f39c006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f39c006bc70 0x7f39c006e130 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39cc007fd0 tx=0x7f39cc007920 comp rx=0 tx=0).stop 2024-02-18T11:48:17.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 msgr2=0x7f39dc15e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 0x7f39dc15e2e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f39d4036040 tx=0x7f39d4004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:17.130 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 shutdown_connections 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f39c006bc70 0x7f39c006e130 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39dc1617e0 0x7f39dc15e2e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39dc15dbf0 0x7f39dc152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 --2- 172.21.15.116:0/3132351497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39dc0cee50 0x7f39dc15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 >> 172.21.15.116:0/3132351497 conn(0x7f39dc0b5480 msgr2=0x7f39dc0b9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 shutdown_connections 2024-02-18T11:48:17.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.133+0000 7f39e388d640 1 -- 172.21.15.116:0/3132351497 wait complete. 2024-02-18T11:48:17.138 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:17.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.248+0000 7f0eb489d640 1 Processor -- start 2024-02-18T11:48:17.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.248+0000 7f0eb489d640 1 -- start start 2024-02-18T11:48:17.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eb489d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0eb015e580 0x7f0eb00cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eb489d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb015dac0 con 0x7f0eb0071a50 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eb489d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb015b000 con 0x7f0eb01547b0 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eb489d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb015b2e0 con 0x7f0eb015e580 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eaed9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0eb015e580 0x7f0eb00cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eaed9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0eb015e580 0x7f0eb00cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39708/0 (socket says 172.21.15.116:39708) 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eaed9d640 1 -- 172.21.15.116:0/234426855 learned_addr learned my addr 172.21.15.116:0/234426855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eaed9d640 1 -- 172.21.15.116:0/234426855 >> v1:172.21.15.143:6789/0 conn(0x7f0eb01547b0 legacy=0x7f0eb015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eaed9d640 1 -- 172.21.15.116:0/234426855 >> v1:172.21.15.116:6789/0 conn(0x7f0eb0071a50 legacy=0x7f0eb0071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eaed9d640 1 -- 172.21.15.116:0/234426855 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb015dd80 con 0x7f0eb015e580 2024-02-18T11:48:17.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.249+0000 7f0eadd9b640 1 -- 172.21.15.116:0/234426855 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170111579 0 0) 0x7f0eb015dac0 con 0x7f0eb0071a50 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.250+0000 7f0eaed9d640 1 --2- 172.21.15.116:0/234426855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb015e580 0x7f0eb00cdc80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009e50 tx=0x7f0ea802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7f46aec7f66be8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.250+0000 7f0eadd9b640 1 -- 172.21.15.116:0/234426855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea802cb20 con 0x7f0eb015e580 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.250+0000 7f0eadd9b640 1 -- 172.21.15.116:0/234426855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea802cc80 con 0x7f0eb015e580 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.250+0000 7f0eadd9b640 1 -- 172.21.15.116:0/234426855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80343f0 con 0x7f0eb015e580 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 -- 172.21.15.116:0/234426855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb015e580 msgr2=0x7f0eb00cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 --2- 172.21.15.116:0/234426855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb015e580 0x7f0eb00cdc80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009e50 tx=0x7f0ea802bf80 comp rx=0 tx=0).stop 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 -- 172.21.15.116:0/234426855 shutdown_connections 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 --2- 172.21.15.116:0/234426855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb015e580 0x7f0eb00cdc80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 -- 172.21.15.116:0/234426855 >> 172.21.15.116:0/234426855 conn(0x7f0eb00cfd80 msgr2=0x7f0eb00d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 -- 172.21.15.116:0/234426855 shutdown_connections 2024-02-18T11:48:17.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.251+0000 7f0eb489d640 1 -- 172.21.15.116:0/234426855 wait complete. 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.252+0000 7f0eb489d640 1 Processor -- start 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.252+0000 7f0eb489d640 1 -- start start 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 0x7f0eb0099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eb01547b0 0x7f0eb015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0eb015e580 0x7f0eb015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0099f90 con 0x7f0eb0071a50 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb014f1b0 con 0x7f0eb01547b0 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb014c270 con 0x7f0eb015e580 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 0x7f0eb0099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 0x7f0eb0099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:39714/0 (socket says 172.21.15.116:39714) 2024-02-18T11:48:17.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 -- 172.21.15.116:0/1701306761 learned_addr learned my addr 172.21.15.116:0/1701306761 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 -- 172.21.15.116:0/1701306761 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0eb015e580 msgr2=0x7f0eb015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0eb015e580 0x7f0eb015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 -- 172.21.15.116:0/1701306761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eb01547b0 msgr2=0x7f0eb015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eb01547b0 0x7f0eb015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 -- 172.21.15.116:0/1701306761 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8009b30 con 0x7f0eb0071a50 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eae59c640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 0x7f0eb0099a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c4b0 tx=0x7f0ea000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0005480 con 0x7f0eb0071a50 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea00035f0 con 0x7f0eb0071a50 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00039e0 con 0x7f0eb0071a50 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb014c550 con 0x7f0eb0071a50 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.253+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb015c550 con 0x7f0eb0071a50 2024-02-18T11:48:17.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.254+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb0067180 con 0x7f0eb0071a50 2024-02-18T11:48:17.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.255+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f0ea0003b40 con 0x7f0eb0071a50 2024-02-18T11:48:17.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.255+0000 7f0ea6ffd640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0e9006bc70 0x7f0e9006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.255+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f0ea008a150 con 0x7f0eb0071a50 2024-02-18T11:48:17.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.255+0000 7f0ea7fff640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0e9006bc70 0x7f0e9006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.256+0000 7f0ea7fff640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0e9006bc70 0x7f0e9006e130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0eb015abf0 tx=0x7f0e980079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.259+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ea00545a0 con 0x7f0eb0071a50 2024-02-18T11:48:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.434+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0eb0074520 con 0x7f0eb0071a50 2024-02-18T11:48:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.434+0000 7f0ea6ffd640 1 -- 172.21.15.116:0/1701306761 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ea0057c60 con 0x7f0eb0071a50 2024-02-18T11:48:17.437 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0e9006bc70 msgr2=0x7f0e9006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0e9006bc70 0x7f0e9006e130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0eb015abf0 tx=0x7f0e980079b0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 msgr2=0x7f0eb0099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 0x7f0eb0099a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c4b0 tx=0x7f0ea000c970 comp rx=0 tx=0).stop 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 shutdown_connections 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0e9006bc70 0x7f0e9006e130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0eb015e580 0x7f0eb015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0eb01547b0 0x7f0eb015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 --2- 172.21.15.116:0/1701306761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0eb0071a50 0x7f0eb0099a50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 >> 172.21.15.116:0/1701306761 conn(0x7f0eb00cfd80 msgr2=0x7f0eb00af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 shutdown_connections 2024-02-18T11:48:17.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.436+0000 7f0eb489d640 1 -- 172.21.15.116:0/1701306761 wait complete. 2024-02-18T11:48:17.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:17.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.554+0000 7f8224fd0640 1 Processor -- start 2024-02-18T11:48:17.548 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.554+0000 7f8224fd0640 1 -- start start 2024-02-18T11:48:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f8224fd0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f82200cd610 0x7f82200d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f8224fd0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82200d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f8224fd0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8220099a50 con 0x7f822009ad40 2024-02-18T11:48:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f8224fd0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8220099ca0 con 0x7f82200cd610 2024-02-18T11:48:17.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f8224fd0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8220099f80 con 0x7f8220161fd0 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821e59c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f82200cd610 0x7f82200d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821dd9b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f822009ad40 0x7f82201618c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53626/0 (socket says 172.21.15.116:53626) 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821dd9b640 1 -- 172.21.15.116:0/2978860756 learned_addr learned my addr 172.21.15.116:0/2978860756 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821e59c640 1 -- 172.21.15.116:0/2978860756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 msgr2=0x7f82200d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821e59c640 1 --2- 172.21.15.116:0/2978860756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82200d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821e59c640 1 -- 172.21.15.116:0/2978860756 >> v1:172.21.15.116:6789/0 conn(0x7f822009ad40 legacy=0x7f82201618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821d59a640 1 -- 172.21.15.116:0/2978860756 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227568179 0 0) 0x7f8220099a50 con 0x7f822009ad40 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.555+0000 7f821e59c640 1 -- 172.21.15.116:0/2978860756 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822009a260 con 0x7f82200cd610 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.556+0000 7f821e59c640 1 --2- 172.21.15.116:0/2978860756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200d0370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82100097a0 tx=0x7f821002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3261097937e6a099 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.556+0000 7f821d59a640 1 -- 172.21.15.116:0/2978860756 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210030610 con 0x7f82200cd610 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.556+0000 7f821d59a640 1 -- 172.21.15.116:0/2978860756 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8210030770 con 0x7f82200cd610 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.556+0000 7f821d59a640 1 -- 172.21.15.116:0/2978860756 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210030b00 con 0x7f82200cd610 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 -- 172.21.15.116:0/2978860756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 msgr2=0x7f82200d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 --2- 172.21.15.116:0/2978860756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200d0370 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f82100097a0 tx=0x7f821002b960 comp rx=0 tx=0).stop 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 -- 172.21.15.116:0/2978860756 shutdown_connections 2024-02-18T11:48:17.553 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 --2- 172.21.15.116:0/2978860756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82200d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 --2- 172.21.15.116:0/2978860756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200d0370 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 -- 172.21.15.116:0/2978860756 >> 172.21.15.116:0/2978860756 conn(0x7f82200b31f0 msgr2=0x7f82200b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:17.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 -- 172.21.15.116:0/2978860756 shutdown_connections 2024-02-18T11:48:17.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.557+0000 7f8224fd0640 1 -- 172.21.15.116:0/2978860756 wait complete. 2024-02-18T11:48:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.558+0000 7f8224fd0640 1 Processor -- start 2024-02-18T11:48:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.558+0000 7f8224fd0640 1 -- start start 2024-02-18T11:48:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f8224fd0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f822009ad40 0x7f822015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f8224fd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f8224fd0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82201e9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f8224fd0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c5c40 con 0x7f82200cd610 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f8224fd0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c5f20 con 0x7f822009ad40 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f8224fd0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c6200 con 0x7f8220161fd0 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44630/0 (socket says 172.21.15.116:44630) 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 -- 172.21.15.116:0/47852386 learned_addr learned my addr 172.21.15.116:0/47852386 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821e59c640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f822009ad40 0x7f822015ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821ed9d640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82201e9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 -- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 msgr2=0x7f82201e9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82201e9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 -- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f822009ad40 msgr2=0x7f822015ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f822009ad40 0x7f822015ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821dd9b640 1 -- 172.21.15.116:0/47852386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210009400 con 0x7f82200cd610 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.559+0000 7f821e59c640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f822009ad40 0x7f822015ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.560+0000 7f821dd9b640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200b5b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f821400c4d0 tx=0x7f821400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.560+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140074e0 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.560+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214007660 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.560+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82200c64e0 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.560+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214004a20 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.560+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82200b6060 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.561+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8220067840 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.561+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f8214004b80 con 0x7f82200cd610 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.561+0000 7f820effd640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f820006bc70 0x7f820006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.561+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f821408a030 con 0x7f82200cd610 2024-02-18T11:48:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.562+0000 7f821e59c640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f820006bc70 0x7f820006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.562+0000 7f821e59c640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f820006bc70 0x7f820006e130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8210036000 tx=0x7f8210008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.566+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8214054480 con 0x7f82200cd610 2024-02-18T11:48:17.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.738+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82200c6a00 con 0x7f82200cd610 2024-02-18T11:48:17.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.738+0000 7f820effd640 1 -- 172.21.15.116:0/47852386 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82200c6a00 con 0x7f82200cd610 2024-02-18T11:48:17.732 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f820006bc70 msgr2=0x7f820006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f820006bc70 0x7f820006e130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8210036000 tx=0x7f8210008030 comp rx=0 tx=0).stop 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 msgr2=0x7f82200b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200b5b20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f821400c4d0 tx=0x7f821400c990 comp rx=0 tx=0).stop 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 shutdown_connections 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f820006bc70 0x7f820006e130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8220161fd0 0x7f82201e9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82200cd610 0x7f82200b5b20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 --2- 172.21.15.116:0/47852386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f822009ad40 0x7f822015ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 >> 172.21.15.116:0/47852386 conn(0x7f82200b31f0 msgr2=0x7f8220070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:17.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 shutdown_connections 2024-02-18T11:48:17.735 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.740+0000 7f8224fd0640 1 -- 172.21.15.116:0/47852386 wait complete. 2024-02-18T11:48:17.748 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:17.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.859+0000 7ff627f02640 1 Processor -- start 2024-02-18T11:48:17.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.859+0000 7ff627f02640 1 -- start start 2024-02-18T11:48:17.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff627f02640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff62015dbf0 0x7ff6201617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff627f02640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6200b9370 0x7ff6200ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff627f02640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6200d1ee0 con 0x7ff6200cec80 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff627f02640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6200cf110 con 0x7ff6200b9370 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff627f02640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff620161dc0 con 0x7ff62015dbf0 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff62549d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6200b9370 0x7ff6200ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff62549d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6200b9370 0x7ff6200ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44632/0 (socket says 172.21.15.116:44632) 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff62649f640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff6200cec80 0x7ff62006c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53628/0 (socket says 172.21.15.116:53628) 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.860+0000 7ff62549d640 1 -- 172.21.15.116:0/1797111241 learned_addr learned my addr 172.21.15.116:0/1797111241 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff625c9e640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff62015dbf0 0x7ff6201617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff62549d640 1 -- 172.21.15.116:0/1797111241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff62015dbf0 msgr2=0x7ff6201617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff62549d640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff62015dbf0 0x7ff6201617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff62549d640 1 -- 172.21.15.116:0/1797111241 >> v1:172.21.15.116:6789/0 conn(0x7ff6200cec80 legacy=0x7ff62006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff62549d640 1 -- 172.21.15.116:0/1797111241 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6201620a0 con 0x7ff6200b9370 2024-02-18T11:48:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff624c9c640 1 -- 172.21.15.116:0/1797111241 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145843009 0 0) 0x7ff6200d1ee0 con 0x7ff6200cec80 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff625c9e640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff62015dbf0 0x7ff6201617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff62549d640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6200b9370 0x7ff6200ce740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff610009870 tx=0x7ff61002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a70bc6a5540ebe server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff624c9c640 1 -- 172.21.15.116:0/1797111241 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610030840 con 0x7ff6200b9370 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff624c9c640 1 -- 172.21.15.116:0/1797111241 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6100309a0 con 0x7ff6200b9370 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.861+0000 7ff624c9c640 1 -- 172.21.15.116:0/1797111241 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610030d30 con 0x7ff6200b9370 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 -- 172.21.15.116:0/1797111241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6200b9370 msgr2=0x7ff6200ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6200b9370 0x7ff6200ce740 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff610009870 tx=0x7ff61002b980 comp rx=0 tx=0).stop 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 -- 172.21.15.116:0/1797111241 shutdown_connections 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6200b9370 0x7ff6200ce740 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 --2- 172.21.15.116:0/1797111241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff62015dbf0 0x7ff6201617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 -- 172.21.15.116:0/1797111241 >> 172.21.15.116:0/1797111241 conn(0x7ff6200b5480 msgr2=0x7ff6200b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 -- 172.21.15.116:0/1797111241 shutdown_connections 2024-02-18T11:48:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.862+0000 7ff627f02640 1 -- 172.21.15.116:0/1797111241 wait complete. 2024-02-18T11:48:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.863+0000 7ff627f02640 1 Processor -- start 2024-02-18T11:48:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.863+0000 7ff627f02640 1 -- start start 2024-02-18T11:48:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff627f02640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6200b9370 0x7ff620151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff627f02640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff6200cec80 0x7ff620154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff627f02640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 0x7ff620153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff627f02640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff620152bd0 con 0x7ff62015dbf0 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff627f02640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6201f3200 con 0x7ff6200b9370 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff627f02640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff620154c80 con 0x7ff6200cec80 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 0x7ff620153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 0x7ff620153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44646/0 (socket says 172.21.15.116:44646) 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 -- 172.21.15.116:0/2023018019 learned_addr learned my addr 172.21.15.116:0/2023018019 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff625c9e640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6200b9370 0x7ff620151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62549d640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff6200cec80 0x7ff620154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 -- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff6200cec80 msgr2=0x7ff620154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff6200cec80 0x7ff620154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 -- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6200b9370 msgr2=0x7ff620151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6200b9370 0x7ff620151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 -- 172.21.15.116:0/2023018019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100094d0 con 0x7ff62015dbf0 2024-02-18T11:48:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff625c9e640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6200b9370 0x7ff620151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62549d640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff6200cec80 0x7ff620154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.864+0000 7ff62649f640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 0x7ff620153060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff61800c450 tx=0x7ff61800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.865+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180074e0 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.865+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618007660 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.865+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62015e150 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.865+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618004a20 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.865+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff620155240 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.866+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff620067840 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.866+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff618004b80 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.866+0000 7ff6167fc640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff60406bc70 0x7ff60406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.867+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7ff61808a060 con 0x7ff62015dbf0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.867+0000 7ff625c9e640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff60406bc70 0x7ff60406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.867+0000 7ff625c9e640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff60406bc70 0x7ff60406e130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff61c007f40 tx=0x7ff61c007490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:17.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:17.871+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff618054420 con 0x7ff62015dbf0 2024-02-18T11:48:18.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.039+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff620074520 con 0x7ff62015dbf0 2024-02-18T11:48:18.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.040+0000 7ff6167fc640 1 -- 172.21.15.116:0/2023018019 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff6180180a0 con 0x7ff62015dbf0 2024-02-18T11:48:18.033 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:18.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff60406bc70 msgr2=0x7ff60406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff60406bc70 0x7ff60406e130 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff61c007f40 tx=0x7ff61c007490 comp rx=0 tx=0).stop 2024-02-18T11:48:18.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 msgr2=0x7ff620153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 0x7ff620153060 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff61800c450 tx=0x7ff61800c910 comp rx=0 tx=0).stop 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 shutdown_connections 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff60406bc70 0x7ff60406e130 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff62015dbf0 0x7ff620153060 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff6200cec80 0x7ff620154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 --2- 172.21.15.116:0/2023018019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6200b9370 0x7ff620151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 >> 172.21.15.116:0/2023018019 conn(0x7ff6200b5480 msgr2=0x7ff6200cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 shutdown_connections 2024-02-18T11:48:18.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.042+0000 7ff627f02640 1 -- 172.21.15.116:0/2023018019 wait complete. 2024-02-18T11:48:18.047 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:18.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.156+0000 7f8b86e9d640 1 Processor -- start 2024-02-18T11:48:18.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.156+0000 7f8b86e9d640 1 -- start start 2024-02-18T11:48:18.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b86e9d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b8015efb0 0x7f8b800d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b86e9d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800d0a80 con 0x7f8b80099a50 2024-02-18T11:48:18.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b86e9d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800cfe70 con 0x7f8b800cf900 2024-02-18T11:48:18.155 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b86e9d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8015f390 con 0x7f8b8015efb0 2024-02-18T11:48:18.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b8543a640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8b80099a50 0x7f8b800cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53636/0 (socket says 172.21.15.116:53636) 2024-02-18T11:48:18.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b8543a640 1 -- 172.21.15.116:0/4223929561 learned_addr learned my addr 172.21.15.116:0/4223929561 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:18.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218098985 0 0) 0x7f8b800d0a80 con 0x7f8b80099a50 2024-02-18T11:48:18.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.157+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b68003520 con 0x7f8b80099a50 2024-02-18T11:48:18.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b84c39640 1 --2- 172.21.15.116:0/4223929561 >> v2:172.21.15.143:3300/0 conn(0x7f8b8015efb0 0x7f8b800d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3928593280 0 0) 0x7f8b68003520 con 0x7f8b80099a50 2024-02-18T11:48:18.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 >> v2:172.21.15.143:3300/0 conn(0x7f8b8015efb0 msgr2=0x7f8b800d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 --2- 172.21.15.116:0/4223929561 >> v2:172.21.15.143:3300/0 conn(0x7f8b8015efb0 0x7f8b800d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 >> v1:172.21.15.143:6789/0 conn(0x7f8b800cf900 legacy=0x7f8b8006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800d0150 con 0x7f8b80099a50 2024-02-18T11:48:18.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f8b780019f0 con 0x7f8b80099a50 2024-02-18T11:48:18.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 >> v1:172.21.15.116:6789/0 conn(0x7f8b80099a50 legacy=0x7f8b800cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b68003970 0x7f8b68003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680052a0 0x7f8b6800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8015f390 con 0x7f8b680047c0 2024-02-18T11:48:18.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800c150 con 0x7f8b68003970 2024-02-18T11:48:18.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800c310 con 0x7f8b680052a0 2024-02-18T11:48:18.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7ffff640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7ffff640 1 -- 172.21.15.116:0/4223929561 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680052a0 msgr2=0x7f8b6800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7ffff640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680052a0 0x7f8b6800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7ffff640 1 -- 172.21.15.116:0/4223929561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b68003970 msgr2=0x7f8b68003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7ffff640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b68003970 0x7f8b68003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b7ffff640 1 -- 172.21.15.116:0/4223929561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68003520 con 0x7f8b680047c0 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.158+0000 7f8b8543a640 1 -- 172.21.15.116:0/4223929561 reap_dead start 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.159+0000 7f8b7ffff640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b74003d80 tx=0x7f8b680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea088886045a19d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.159+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74032600 con 0x7f8b680047c0 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.159+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7402e050 con 0x7f8b680047c0 2024-02-18T11:48:18.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.159+0000 7f8b7f7fe640 1 -- 172.21.15.116:0/4223929561 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740329e0 con 0x7f8b680047c0 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.159+0000 7f8b86e9d640 1 -- 172.21.15.116:0/4223929561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b680047c0 msgr2=0x7f8b68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.160+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b74003d80 tx=0x7f8b680057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.160+0000 7f8b86e9d640 1 -- 172.21.15.116:0/4223929561 shutdown_connections 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.160+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/4223929561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.160+0000 7f8b86e9d640 1 -- 172.21.15.116:0/4223929561 >> 172.21.15.116:0/4223929561 conn(0x7f8b800d1790 msgr2=0x7f8b800d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.160+0000 7f8b86e9d640 1 -- 172.21.15.116:0/4223929561 shutdown_connections 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.160+0000 7f8b86e9d640 1 -- 172.21.15.116:0/4223929561 wait complete. 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.161+0000 7f8b86e9d640 1 Processor -- start 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.161+0000 7f8b86e9d640 1 -- start start 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680047c0 0x7f8b8015ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8014ec00 0x7f8b80161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 0x7f8b800b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800b71a0 con 0x7f8b800b67b0 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b800c5ec0 con 0x7f8b8014ec00 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80162070 con 0x7f8b680047c0 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 0x7f8b800b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 0x7f8b800b6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44666/0 (socket says 172.21.15.116:44666) 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 -- 172.21.15.116:0/1584651666 learned_addr learned my addr 172.21.15.116:0/1584651666 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 -- 172.21.15.116:0/1584651666 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680047c0 msgr2=0x7f8b8015ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680047c0 0x7f8b8015ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 -- 172.21.15.116:0/1584651666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8014ec00 msgr2=0x7f8b80161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8014ec00 0x7f8b80161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 -- 172.21.15.116:0/1584651666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80162350 con 0x7f8b800b67b0 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b8543a640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 0x7f8b800b6c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8b800c5290 tx=0x7f8b7800ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800a6d0 con 0x7f8b800b67b0 2024-02-18T11:48:18.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7800a850 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800ac40 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b80162630 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.162+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b80161d40 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.163+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b80067840 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.164+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f8b78007070 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.164+0000 7f8b7d7fa640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8b6c06bca0 0x7f8b6c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.164+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f8b7808a8d0 con 0x7f8b800b67b0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.164+0000 7f8b84c39640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8b6c06bca0 0x7f8b6c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.165+0000 7f8b84c39640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8b6c06bca0 0x7f8b6c06e160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b8006a410 tx=0x7f8b70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.169+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b78054c90 con 0x7f8b800b67b0 2024-02-18T11:48:18.334 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.340+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b800c6350 con 0x7f8b800b67b0 2024-02-18T11:48:18.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.341+0000 7f8b7d7fa640 1 -- 172.21.15.116:0/1584651666 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f8b800c6350 con 0x7f8b800b67b0 2024-02-18T11:48:18.335 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:18.335 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:18.337 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.343+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8b6c06bca0 msgr2=0x7f8b6c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.343+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8b6c06bca0 0x7f8b6c06e160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b8006a410 tx=0x7f8b70008040 comp rx=0 tx=0).stop 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 msgr2=0x7f8b800b6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 0x7f8b800b6c60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8b800c5290 tx=0x7f8b7800ce80 comp rx=0 tx=0).stop 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 shutdown_connections 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8b6c06bca0 0x7f8b6c06e160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b800b67b0 0x7f8b800b6c60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8014ec00 0x7f8b80161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 --2- 172.21.15.116:0/1584651666 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b680047c0 0x7f8b8015ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 >> 172.21.15.116:0/1584651666 conn(0x7f8b800d1790 msgr2=0x7f8b8006c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 shutdown_connections 2024-02-18T11:48:18.338 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.344+0000 7f8b86e9d640 1 -- 172.21.15.116:0/1584651666 wait complete. 2024-02-18T11:48:18.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:18.349 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:18.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.459+0000 7f5fd89d0640 1 Processor -- start 2024-02-18T11:48:18.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.459+0000 7f5fd89d0640 1 -- start start 2024-02-18T11:48:18.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.460+0000 7f5fd89d0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fd4144090 0x7f5fd40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.460+0000 7f5fd89d0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5fd406ea40 0x7f5fd415e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.460+0000 7f5fd89d0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd40b2a50 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.460+0000 7f5fd89d0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd415b090 con 0x7f5fd406ea40 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd89d0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd415b370 con 0x7f5fd4144090 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5fd406ea40 0x7f5fd415e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5fd406ea40 0x7f5fd415e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44678/0 (socket says 172.21.15.116:44678) 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 -- 172.21.15.116:0/1961520262 learned_addr learned my addr 172.21.15.116:0/1961520262 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd259c640 1 --2- 172.21.15.116:0/1961520262 >> v2:172.21.15.143:3300/0 conn(0x7f5fd4144090 0x7f5fd40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 -- 172.21.15.116:0/1961520262 >> v2:172.21.15.143:3300/0 conn(0x7f5fd4144090 msgr2=0x7f5fd40b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 --2- 172.21.15.116:0/1961520262 >> v2:172.21.15.143:3300/0 conn(0x7f5fd4144090 0x7f5fd40b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 -- 172.21.15.116:0/1961520262 >> v1:172.21.15.143:6789/0 conn(0x7f5fd40cc9e0 legacy=0x7f5fd406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd2d9d640 1 -- 172.21.15.116:0/1961520262 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd415b650 con 0x7f5fd406ea40 2024-02-18T11:48:18.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.461+0000 7f5fd259c640 1 --2- 172.21.15.116:0/1961520262 >> v2:172.21.15.143:3300/0 conn(0x7f5fd4144090 0x7f5fd40b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd2d9d640 1 --2- 172.21.15.116:0/1961520262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd406ea40 0x7f5fd415e610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40b4ce0 tx=0x7f5fc402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe83f38317ca2a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd159a640 1 -- 172.21.15.116:0/1961520262 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc402ee00 con 0x7f5fd406ea40 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd159a640 1 -- 172.21.15.116:0/1961520262 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc4036440 con 0x7f5fd406ea40 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd159a640 1 -- 172.21.15.116:0/1961520262 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40367d0 con 0x7f5fd406ea40 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 -- 172.21.15.116:0/1961520262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd406ea40 msgr2=0x7f5fd415e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/1961520262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd406ea40 0x7f5fd415e610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40b4ce0 tx=0x7f5fc402e050 comp rx=0 tx=0).stop 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 -- 172.21.15.116:0/1961520262 shutdown_connections 2024-02-18T11:48:18.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/1961520262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd406ea40 0x7f5fd415e610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/1961520262 >> v2:172.21.15.143:3300/0 conn(0x7f5fd4144090 0x7f5fd40b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 -- 172.21.15.116:0/1961520262 >> 172.21.15.116:0/1961520262 conn(0x7f5fd4067000 msgr2=0x7f5fd406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 -- 172.21.15.116:0/1961520262 shutdown_connections 2024-02-18T11:48:18.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.462+0000 7f5fd89d0640 1 -- 172.21.15.116:0/1961520262 wait complete. 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 Processor -- start 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 -- start start 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5fd406ea40 0x7f5fd415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 0x7f5fd413def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4144090 0x7f5fd41f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40ac8c0 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41f7100 con 0x7f5fd4144090 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.464+0000 7f5fd89d0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41f73e0 con 0x7f5fd406ea40 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 0x7f5fd413def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 0x7f5fd413def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44680/0 (socket says 172.21.15.116:44680) 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 -- 172.21.15.116:0/2730222101 learned_addr learned my addr 172.21.15.116:0/2730222101 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd2d9d640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4144090 0x7f5fd41f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 -- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5fd406ea40 msgr2=0x7f5fd415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5fd406ea40 0x7f5fd415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 -- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4144090 msgr2=0x7f5fd41f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4144090 0x7f5fd41f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 -- 172.21.15.116:0/2730222101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc400b770 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd2d9d640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4144090 0x7f5fd41f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd1d9b640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 0x7f5fd413def0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c7c0 tx=0x7f5fc800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800aa50 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc800abb0 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd41f76c0 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8012420 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.465+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd413e430 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.466+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd4074520 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.467+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f5fc8012580 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.467+0000 7f5fc2ffd640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5fb406bc70 0x7f5fb406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.462 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.467+0000 7f5fd259c640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5fb406bc70 0x7f5fb406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.467+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f5fc808ab20 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.467+0000 7f5fd259c640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5fb406bc70 0x7f5fb406e130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc006fd0 tx=0x7f5fcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.465 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.471+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fc8054ee0 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.649+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fd406ab60 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.650+0000 7f5fc2ffd640 1 -- 172.21.15.116:0/2730222101 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fc80585a0 con 0x7f5fd40cc9e0 2024-02-18T11:48:18.643 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5fb406bc70 msgr2=0x7f5fb406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5fb406bc70 0x7f5fb406e130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc006fd0 tx=0x7f5fcc008040 comp rx=0 tx=0).stop 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 msgr2=0x7f5fd413def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 0x7f5fd413def0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c7c0 tx=0x7f5fc800cc80 comp rx=0 tx=0).stop 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 shutdown_connections 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5fb406bc70 0x7f5fb406e130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd4144090 0x7f5fd41f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5fd40cc9e0 0x7f5fd413def0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 --2- 172.21.15.116:0/2730222101 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5fd406ea40 0x7f5fd415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 >> 172.21.15.116:0/2730222101 conn(0x7f5fd4067000 msgr2=0x7f5fd406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.651+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 shutdown_connections 2024-02-18T11:48:18.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.652+0000 7f5fd89d0640 1 -- 172.21.15.116:0/2730222101 wait complete. 2024-02-18T11:48:18.656 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:18.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.766+0000 7f739dc1b640 1 Processor -- start 2024-02-18T11:48:18.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.766+0000 7f739dc1b640 1 -- start start 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f739dc1b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f739815dc90 0x7f73980ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f739dc1b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f739806f300 0x7f73980b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f739dc1b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73980cf980 con 0x7f73980cf3e0 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f739dc1b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739815e070 con 0x7f739806f300 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f739dc1b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739806ab60 con 0x7f739815dc90 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f7397fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f739806f300 0x7f73980b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f7397fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f739806f300 0x7f73980b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44692/0 (socket says 172.21.15.116:44692) 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f7397fff640 1 -- 172.21.15.116:0/3455434266 learned_addr learned my addr 172.21.15.116:0/3455434266 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.767+0000 7f73977fe640 1 --2- 172.21.15.116:0/3455434266 >> v2:172.21.15.143:3300/0 conn(0x7f739815dc90 0x7f73980ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f7397fff640 1 -- 172.21.15.116:0/3455434266 >> v2:172.21.15.143:3300/0 conn(0x7f739815dc90 msgr2=0x7f73980ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:18.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f7397fff640 1 --2- 172.21.15.116:0/3455434266 >> v2:172.21.15.143:3300/0 conn(0x7f739815dc90 0x7f73980ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f7397fff640 1 -- 172.21.15.116:0/3455434266 >> v1:172.21.15.143:6789/0 conn(0x7f73980cf3e0 legacy=0x7f739806ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f7397fff640 1 -- 172.21.15.116:0/3455434266 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739806ae40 con 0x7f739806f300 2024-02-18T11:48:18.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f7397fff640 1 --2- 172.21.15.116:0/3455434266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739806f300 0x7f73980b2db0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7398066fc0 tx=0x7f739002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39350e0eaf887b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f73967fc640 1 -- 172.21.15.116:0/3455434266 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739002cb20 con 0x7f739806f300 2024-02-18T11:48:18.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f73967fc640 1 -- 172.21.15.116:0/3455434266 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739002cc80 con 0x7f739806f300 2024-02-18T11:48:18.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.768+0000 7f73967fc640 1 -- 172.21.15.116:0/3455434266 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73900343f0 con 0x7f739806f300 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 -- 172.21.15.116:0/3455434266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739806f300 msgr2=0x7f73980b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 --2- 172.21.15.116:0/3455434266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739806f300 0x7f73980b2db0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7398066fc0 tx=0x7f739002bf80 comp rx=0 tx=0).stop 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 -- 172.21.15.116:0/3455434266 shutdown_connections 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 --2- 172.21.15.116:0/3455434266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739806f300 0x7f73980b2db0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 --2- 172.21.15.116:0/3455434266 >> v2:172.21.15.143:3300/0 conn(0x7f739815dc90 0x7f73980ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 -- 172.21.15.116:0/3455434266 >> 172.21.15.116:0/3455434266 conn(0x7f73980cfd80 msgr2=0x7f73980d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 -- 172.21.15.116:0/3455434266 shutdown_connections 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.769+0000 7f739dc1b640 1 -- 172.21.15.116:0/3455434266 wait complete. 2024-02-18T11:48:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.770+0000 7f739dc1b640 1 Processor -- start 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.770+0000 7f739dc1b640 1 -- start start 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f739dc1b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f739806f300 0x7f739814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f739dc1b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73980cf3e0 0x7f73980b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f739dc1b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 0x7f73980b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f739dc1b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739815edb0 con 0x7f739815dc90 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f739dc1b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739815e1e0 con 0x7f73980cf3e0 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f739dc1b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739815e480 con 0x7f739806f300 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 0x7f73980b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 0x7f73980b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44694/0 (socket says 172.21.15.116:44694) 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 -- 172.21.15.116:0/2634524416 learned_addr learned my addr 172.21.15.116:0/2634524416 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7396ffd640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73980cf3e0 0x7f73980b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f73977fe640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f739806f300 0x7f739814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 -- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f739806f300 msgr2=0x7f739814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f739806f300 0x7f739814c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 -- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73980cf3e0 msgr2=0x7f73980b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73980cf3e0 0x7f73980b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.771+0000 7f7397fff640 1 -- 172.21.15.116:0/2634524416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7390009b30 con 0x7f739815dc90 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.772+0000 7f7397fff640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 0x7f73980b4070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7390009f40 tx=0x7f7390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.772+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390009510 con 0x7f739815dc90 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.772+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739002c920 con 0x7f739815dc90 2024-02-18T11:48:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.772+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739815e760 con 0x7f739815dc90 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.772+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739003e690 con 0x7f739815dc90 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.772+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739814cda0 con 0x7f739815dc90 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.773+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7398067180 con 0x7f739815dc90 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.773+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f739003e7f0 con 0x7f739815dc90 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.774+0000 7f737ffff640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f738406bc70 0x7f738406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.774+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f73900ad520 con 0x7f739815dc90 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.774+0000 7f73977fe640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f738406bc70 0x7f738406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.774+0000 7f73977fe640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f738406bc70 0x7f738406e130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7388009aa0 tx=0x7f7388009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:18.774 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.778+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7390077910 con 0x7f739815dc90 2024-02-18T11:48:18.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.949+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7398074520 con 0x7f739815dc90 2024-02-18T11:48:18.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.949+0000 7f737ffff640 1 -- 172.21.15.116:0/2634524416 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f739007afd0 con 0x7f739815dc90 2024-02-18T11:48:18.943 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f738406bc70 msgr2=0x7f738406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f738406bc70 0x7f738406e130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7388009aa0 tx=0x7f7388009190 comp rx=0 tx=0).stop 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 msgr2=0x7f73980b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 0x7f73980b4070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7390009f40 tx=0x7f7390004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 shutdown_connections 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f738406bc70 0x7f738406e130 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f739815dc90 0x7f73980b4070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73980cf3e0 0x7f73980b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.951+0000 7f739dc1b640 1 --2- 172.21.15.116:0/2634524416 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f739806f300 0x7f739814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:18.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.952+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 >> 172.21.15.116:0/2634524416 conn(0x7f73980cfd80 msgr2=0x7f73980cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:18.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.952+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 shutdown_connections 2024-02-18T11:48:18.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:18.952+0000 7f739dc1b640 1 -- 172.21.15.116:0/2634524416 wait complete. 2024-02-18T11:48:18.956 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:19.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.065+0000 7f14ff4c0640 1 Processor -- start 2024-02-18T11:48:19.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.065+0000 7f14ff4c0640 1 -- start start 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14ff4c0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14f8144090 0x7f14f80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14ff4c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f14f806ea40 0x7f14f815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14ff4c0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f80b2a50 con 0x7f14f80cc9e0 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14ff4c0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f815b090 con 0x7f14f806ea40 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14ff4c0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f815b370 con 0x7f14f8144090 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14fda5d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f14f806ea40 0x7f14f815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14fda5d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f14f806ea40 0x7f14f815e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44696/0 (socket says 172.21.15.116:44696) 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14fda5d640 1 -- 172.21.15.116:0/890518720 learned_addr learned my addr 172.21.15.116:0/890518720 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14fda5d640 1 -- 172.21.15.116:0/890518720 >> v2:172.21.15.143:3300/0 conn(0x7f14f8144090 msgr2=0x7f14f80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:19.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.066+0000 7f14fd25c640 1 --2- 172.21.15.116:0/890518720 >> v2:172.21.15.143:3300/0 conn(0x7f14f8144090 0x7f14f80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14fda5d640 1 --2- 172.21.15.116:0/890518720 >> v2:172.21.15.143:3300/0 conn(0x7f14f8144090 0x7f14f80b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14fda5d640 1 -- 172.21.15.116:0/890518720 >> v1:172.21.15.143:6789/0 conn(0x7f14f80cc9e0 legacy=0x7f14f806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:19.061 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14fda5d640 1 -- 172.21.15.116:0/890518720 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f815b650 con 0x7f14f806ea40 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14fda5d640 1 --2- 172.21.15.116:0/890518720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f806ea40 0x7f14f815e610 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14f80b4ce0 tx=0x7f14f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8d94373fc5abd8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14effff640 1 -- 172.21.15.116:0/890518720 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f00317e0 con 0x7f14f806ea40 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14effff640 1 -- 172.21.15.116:0/890518720 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f0031940 con 0x7f14f806ea40 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.067+0000 7f14effff640 1 -- 172.21.15.116:0/890518720 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0031cd0 con 0x7f14f806ea40 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 -- 172.21.15.116:0/890518720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f806ea40 msgr2=0x7f14f815e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/890518720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f806ea40 0x7f14f815e610 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f14f80b4ce0 tx=0x7f14f002b880 comp rx=0 tx=0).stop 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 -- 172.21.15.116:0/890518720 shutdown_connections 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/890518720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f806ea40 0x7f14f815e610 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/890518720 >> v2:172.21.15.143:3300/0 conn(0x7f14f8144090 0x7f14f80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.062 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 -- 172.21.15.116:0/890518720 >> 172.21.15.116:0/890518720 conn(0x7f14f8067000 msgr2=0x7f14f806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 -- 172.21.15.116:0/890518720 shutdown_connections 2024-02-18T11:48:19.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.068+0000 7f14ff4c0640 1 -- 172.21.15.116:0/890518720 wait complete. 2024-02-18T11:48:19.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.069+0000 7f14ff4c0640 1 Processor -- start 2024-02-18T11:48:19.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.069+0000 7f14ff4c0640 1 -- start start 2024-02-18T11:48:19.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14ff4c0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f14f806ea40 0x7f14f80c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14ff4c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 0x7f14f813e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14ff4c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14f8144090 0x7f14f80a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14ff4c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f80c4560 con 0x7f14f80cc9e0 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14ff4c0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f80c5430 con 0x7f14f8144090 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14ff4c0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f813eef0 con 0x7f14f806ea40 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 0x7f14f813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 0x7f14f813e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44706/0 (socket says 172.21.15.116:44706) 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 -- 172.21.15.116:0/1648911121 learned_addr learned my addr 172.21.15.116:0/1648911121 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fda5d640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14f8144090 0x7f14f80a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fd25c640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f14f806ea40 0x7f14f80c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 -- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f14f806ea40 msgr2=0x7f14f80c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f14f806ea40 0x7f14f80c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 -- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14f8144090 msgr2=0x7f14f80a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14f8144090 0x7f14f80a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fca5b640 1 -- 172.21.15.116:0/1648911121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00094d0 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.070+0000 7f14fd25c640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f14f806ea40 0x7f14f80c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.071+0000 7f14fca5b640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 0x7f14f813e9b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c460 tx=0x7f14e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.071+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011790 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.071+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e8011930 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.071+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f81f4550 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.071+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011ce0 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.071+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f80c4a50 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.072+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f80cbe40 con 0x7f14f80cc9e0 2024-02-18T11:48:19.067 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.072+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f14e801c460 con 0x7f14f80cc9e0 2024-02-18T11:48:19.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.073+0000 7f14edffb640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f14dc06bca0 0x7f14dc06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.073+0000 7f14fd25c640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f14dc06bca0 0x7f14dc06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.073+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f14e808a9b0 con 0x7f14f80cc9e0 2024-02-18T11:48:19.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.073+0000 7f14fd25c640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f14dc06bca0 0x7f14dc06e160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14f400e4a0 tx=0x7f14f4007650 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.070 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.077+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14e8054d70 con 0x7f14f80cc9e0 2024-02-18T11:48:19.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.249+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14f8074520 con 0x7f14f80cc9e0 2024-02-18T11:48:19.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.250+0000 7f14edffb640 1 -- 172.21.15.116:0/1648911121 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14e8058430 con 0x7f14f80cc9e0 2024-02-18T11:48:19.243 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:19.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f14dc06bca0 msgr2=0x7f14dc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f14dc06bca0 0x7f14dc06e160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14f400e4a0 tx=0x7f14f4007650 comp rx=0 tx=0).stop 2024-02-18T11:48:19.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 msgr2=0x7f14f813e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 0x7f14f813e9b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c460 tx=0x7f14e800c920 comp rx=0 tx=0).stop 2024-02-18T11:48:19.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 shutdown_connections 2024-02-18T11:48:19.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f14dc06bca0 0x7f14dc06e160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14f8144090 0x7f14f80a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14f80cc9e0 0x7f14f813e9b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 --2- 172.21.15.116:0/1648911121 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f14f806ea40 0x7f14f80c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 >> 172.21.15.116:0/1648911121 conn(0x7f14f8067000 msgr2=0x7f14f806e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 shutdown_connections 2024-02-18T11:48:19.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.252+0000 7f14ff4c0640 1 -- 172.21.15.116:0/1648911121 wait complete. 2024-02-18T11:48:19.256 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:19.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.367+0000 7f4aeabb5640 1 Processor -- start 2024-02-18T11:48:19.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.368+0000 7f4aeabb5640 1 -- start start 2024-02-18T11:48:19.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.368+0000 7f4aeabb5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae415dc90 0x7f4ae40ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.368+0000 7f4aeabb5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ae406f300 0x7f4ae40b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.368+0000 7f4aeabb5640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae40cf980 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.368+0000 7f4aeabb5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae415e070 con 0x7f4ae406f300 2024-02-18T11:48:19.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.368+0000 7f4aeabb5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae406ab60 con 0x7f4ae415dc90 2024-02-18T11:48:19.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae8951640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae415dc90 0x7f4ae40ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae9152640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ae406f300 0x7f4ae40b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae8951640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae415dc90 0x7f4ae40ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:54790/0 (socket says 172.21.15.116:54790) 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4adbfff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4ae40cf3e0 0x7f4ae406ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:43598/0 (socket says 172.21.15.116:43598) 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae8951640 1 -- 172.21.15.116:0/3456042579 learned_addr learned my addr 172.21.15.116:0/3456042579 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae9152640 1 -- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae415dc90 msgr2=0x7f4ae40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae9152640 1 --2- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae415dc90 0x7f4ae40ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae9152640 1 -- 172.21.15.116:0/3456042579 >> v1:172.21.15.143:6789/0 conn(0x7f4ae40cf3e0 legacy=0x7f4ae406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.369+0000 7f4ae9152640 1 -- 172.21.15.116:0/3456042579 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae406ae40 con 0x7f4ae406f300 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4ae9152640 1 --2- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae406f300 0x7f4ae40b2db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009d90 tx=0x7f4adc02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a150911fdb8fc443 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4adb7fe640 1 -- 172.21.15.116:0/3456042579 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc02cc50 con 0x7f4ae406f300 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4adb7fe640 1 -- 172.21.15.116:0/3456042579 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc02cdb0 con 0x7f4ae406f300 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4adb7fe640 1 -- 172.21.15.116:0/3456042579 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc034660 con 0x7f4ae406f300 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 -- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae406f300 msgr2=0x7f4ae40b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae406f300 0x7f4ae40b2db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009d90 tx=0x7f4adc02bea0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 -- 172.21.15.116:0/3456042579 shutdown_connections 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae406f300 0x7f4ae40b2db0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/3456042579 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae415dc90 0x7f4ae40ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 -- 172.21.15.116:0/3456042579 >> 172.21.15.116:0/3456042579 conn(0x7f4ae40cfd80 msgr2=0x7f4ae40d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.370+0000 7f4aeabb5640 1 -- 172.21.15.116:0/3456042579 shutdown_connections 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.371+0000 7f4aeabb5640 1 -- 172.21.15.116:0/3456042579 wait complete. 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 Processor -- start 2024-02-18T11:48:19.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 -- start start 2024-02-18T11:48:19.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae406f300 0x7f4ae415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 0x7f4ae414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae415dc90 0x7f4ae41583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae414ad10 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae415f2e0 con 0x7f4ae415dc90 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.372+0000 7f4aeabb5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4160c30 con 0x7f4ae406f300 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 0x7f4ae414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 0x7f4ae414a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44718/0 (socket says 172.21.15.116:44718) 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 -- 172.21.15.116:0/339801949 learned_addr learned my addr 172.21.15.116:0/339801949 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4ae9152640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae415dc90 0x7f4ae41583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4ae8951640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae406f300 0x7f4ae415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 -- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae406f300 msgr2=0x7f4ae415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae406f300 0x7f4ae415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 -- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae415dc90 msgr2=0x7f4ae41583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae415dc90 0x7f4ae41583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 -- 172.21.15.116:0/339801949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc009a70 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4adbfff640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 0x7f4ae414a7d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c4d0 tx=0x7f4ad400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4007480 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad40075e0 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40049e0 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae415ec90 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.373+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae4160f10 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.374+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae4067180 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.375+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4ad4004b40 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.375+0000 7f4ad97fa640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4ad006bc70 0x7f4ad006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.375+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f4ad4089ff0 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.375+0000 7f4ae8951640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4ad006bc70 0x7f4ad006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.375+0000 7f4ae8951640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4ad006bc70 0x7f4ad006e130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009ac0 tx=0x7f4ae0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.373 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.379+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ad40544f0 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.549+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ae40725d0 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.550+0000 7f4ad97fa640 1 -- 172.21.15.116:0/339801949 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4ad4057bb0 con 0x7f4ae40cf3e0 2024-02-18T11:48:19.543 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4ad006bc70 msgr2=0x7f4ad006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4ad006bc70 0x7f4ad006e130 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009ac0 tx=0x7f4ae0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 msgr2=0x7f4ae414a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 0x7f4ae414a7d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400c4d0 tx=0x7f4ad400c990 comp rx=0 tx=0).stop 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 shutdown_connections 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4ad006bc70 0x7f4ad006e130 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.546 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae415dc90 0x7f4ae41583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ae40cf3e0 0x7f4ae414a7d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 --2- 172.21.15.116:0/339801949 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4ae406f300 0x7f4ae415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 >> 172.21.15.116:0/339801949 conn(0x7f4ae40cfd80 msgr2=0x7f4ae40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 shutdown_connections 2024-02-18T11:48:19.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.552+0000 7f4aeabb5640 1 -- 172.21.15.116:0/339801949 wait complete. 2024-02-18T11:48:19.556 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:19.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.663+0000 7f99e86c0640 1 Processor -- start 2024-02-18T11:48:19.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.664+0000 7f99e86c0640 1 -- start start 2024-02-18T11:48:19.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.664+0000 7f99e86c0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99e015a860 0x7f99e00cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.664+0000 7f99e86c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99e006ea20 0x7f99e00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.664+0000 7f99e86c0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00b02e0 con 0x7f99e00cd1f0 2024-02-18T11:48:19.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.664+0000 7f99e86c0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00af6d0 con 0x7f99e006ea20 2024-02-18T11:48:19.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.664+0000 7f99e86c0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00af9b0 con 0x7f99e015a860 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e6c5d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f99e006ea20 0x7f99e00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e5c5b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f99e00cd1f0 0x7f99e006e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53646/0 (socket says 172.21.15.116:53646) 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e5c5b640 1 -- 172.21.15.116:0/1861265742 learned_addr learned my addr 172.21.15.116:0/1861265742 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e645c640 1 --2- 172.21.15.116:0/1861265742 >> v2:172.21.15.143:3300/0 conn(0x7f99e015a860 0x7f99e00cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e6c5d640 1 -- 172.21.15.116:0/1861265742 >> v2:172.21.15.143:3300/0 conn(0x7f99e015a860 msgr2=0x7f99e00cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e6c5d640 1 --2- 172.21.15.116:0/1861265742 >> v2:172.21.15.143:3300/0 conn(0x7f99e015a860 0x7f99e00cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e6c5d640 1 -- 172.21.15.116:0/1861265742 >> v1:172.21.15.116:6789/0 conn(0x7f99e00cd1f0 legacy=0x7f99e006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e6c5d640 1 -- 172.21.15.116:0/1861265742 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00afc90 con 0x7f99e006ea20 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e545a640 1 -- 172.21.15.116:0/1861265742 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912423641 0 0) 0x7f99e00b02e0 con 0x7f99e00cd1f0 2024-02-18T11:48:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.665+0000 7f99e645c640 1 --2- 172.21.15.116:0/1861265742 >> v2:172.21.15.143:3300/0 conn(0x7f99e015a860 0x7f99e00cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e6c5d640 1 --2- 172.21.15.116:0/1861265742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e00b0770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99e006ef20 tx=0x7f99dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3acc8d35ac4975 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e545a640 1 -- 172.21.15.116:0/1861265742 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc031830 con 0x7f99e006ea20 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e545a640 1 -- 172.21.15.116:0/1861265742 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc031990 con 0x7f99e006ea20 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e545a640 1 -- 172.21.15.116:0/1861265742 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc031d20 con 0x7f99e006ea20 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 -- 172.21.15.116:0/1861265742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 msgr2=0x7f99e00b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 --2- 172.21.15.116:0/1861265742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e00b0770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f99e006ef20 tx=0x7f99dc02b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 -- 172.21.15.116:0/1861265742 shutdown_connections 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 --2- 172.21.15.116:0/1861265742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e00b0770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 --2- 172.21.15.116:0/1861265742 >> v2:172.21.15.143:3300/0 conn(0x7f99e015a860 0x7f99e00cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 -- 172.21.15.116:0/1861265742 >> 172.21.15.116:0/1861265742 conn(0x7f99e0066f20 msgr2=0x7f99e006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.666+0000 7f99e86c0640 1 -- 172.21.15.116:0/1861265742 shutdown_connections 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.667+0000 7f99e86c0640 1 -- 172.21.15.116:0/1861265742 wait complete. 2024-02-18T11:48:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 Processor -- start 2024-02-18T11:48:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 -- start start 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e01e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99e00cd1f0 0x7f99e0157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99e015a860 0x7f99e01f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0158f30 con 0x7f99e006ea20 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01580a0 con 0x7f99e00cd1f0 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e86c0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0158320 con 0x7f99e015a860 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e645c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e01e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e645c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e01e49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44734/0 (socket says 172.21.15.116:44734) 2024-02-18T11:48:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.668+0000 7f99e645c640 1 -- 172.21.15.116:0/350634569 learned_addr learned my addr 172.21.15.116:0/350634569 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e645c640 1 -- 172.21.15.116:0/350634569 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99e015a860 msgr2=0x7f99e01f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e645c640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99e015a860 0x7f99e01f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e645c640 1 -- 172.21.15.116:0/350634569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99e00cd1f0 msgr2=0x7f99e0157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e645c640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99e00cd1f0 0x7f99e0157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e645c640 1 -- 172.21.15.116:0/350634569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc009400 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e645c640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e01e49b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f99d8007f20 tx=0x7f99d800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d800dee0 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d800f040 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e0158600 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003db0 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.669+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e0158bc0 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.670+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0067510 con 0x7f99e006ea20 2024-02-18T11:48:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.670+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f99d8004030 con 0x7f99e006ea20 2024-02-18T11:48:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.671+0000 7f99d6ffd640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99c406bca0 0x7f99c406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.671+0000 7f99e5c5b640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99c406bca0 0x7f99c406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.671+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f99d808acc0 con 0x7f99e006ea20 2024-02-18T11:48:19.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.671+0000 7f99e5c5b640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99c406bca0 0x7f99c406e160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99d0007fd0 tx=0x7f99d0007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.675+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99d80551c0 con 0x7f99e006ea20 2024-02-18T11:48:19.841 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.848+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99e0074520 con 0x7f99e006ea20 2024-02-18T11:48:19.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.849+0000 7f99d6ffd640 1 -- 172.21.15.116:0/350634569 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f99d8058880 con 0x7f99e006ea20 2024-02-18T11:48:19.843 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:19.843 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99c406bca0 msgr2=0x7f99c406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99c406bca0 0x7f99c406e160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99d0007fd0 tx=0x7f99d0007900 comp rx=0 tx=0).stop 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 msgr2=0x7f99e01e49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e01e49b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f99d8007f20 tx=0x7f99d800d560 comp rx=0 tx=0).stop 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 shutdown_connections 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f99c406bca0 0x7f99c406e160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f99e015a860 0x7f99e01f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99e00cd1f0 0x7f99e0157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 --2- 172.21.15.116:0/350634569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99e006ea20 0x7f99e01e49b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 >> 172.21.15.116:0/350634569 conn(0x7f99e0066f20 msgr2=0x7f99e0157000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 shutdown_connections 2024-02-18T11:48:19.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.851+0000 7f99e86c0640 1 -- 172.21.15.116:0/350634569 wait complete. 2024-02-18T11:48:19.856 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:19.857 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:19.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.967+0000 7f8ea6327640 1 Processor -- start 2024-02-18T11:48:19.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.967+0000 7f8ea6327640 1 -- start start 2024-02-18T11:48:19.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea6327640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ea0145230 0x7f8ea015be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea6327640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea015c3d0 0x7f8ea00711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea6327640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8ea0071700 0x7f8ea006d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea6327640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea015d050 con 0x7f8ea0145230 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea6327640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea0145610 con 0x7f8ea0071700 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea6327640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0071b50 con 0x7f8ea015c3d0 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ea0145230 0x7f8ea015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ea0145230 0x7f8ea015be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44750/0 (socket says 172.21.15.116:44750) 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 -- 172.21.15.116:0/2301616469 learned_addr learned my addr 172.21.15.116:0/2301616469 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea48c4640 1 --2- 172.21.15.116:0/2301616469 >> v2:172.21.15.143:3300/0 conn(0x7f8ea0071700 0x7f8ea006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9f7fe640 1 --2- 172.21.15.116:0/2301616469 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea015c3d0 0x7f8ea00711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 -- 172.21.15.116:0/2301616469 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea015c3d0 msgr2=0x7f8ea00711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 --2- 172.21.15.116:0/2301616469 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea015c3d0 0x7f8ea00711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 -- 172.21.15.116:0/2301616469 >> v2:172.21.15.143:3300/0 conn(0x7f8ea0071700 msgr2=0x7f8ea006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 --2- 172.21.15.116:0/2301616469 >> v2:172.21.15.143:3300/0 conn(0x7f8ea0071700 0x7f8ea006d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8e9ffff640 1 -- 172.21.15.116:0/2301616469 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea015c960 con 0x7f8ea0145230 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.968+0000 7f8ea48c4640 1 --2- 172.21.15.116:0/2301616469 >> v2:172.21.15.143:3300/0 conn(0x7f8ea0071700 0x7f8ea006d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:19.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.969+0000 7f8e9ffff640 1 --2- 172.21.15.116:0/2301616469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea0145230 0x7f8ea015be90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ea006f320 tx=0x7f8e9002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186a871791ec6dfd server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.969+0000 7f8e9effd640 1 -- 172.21.15.116:0/2301616469 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900317e0 con 0x7f8ea0145230 2024-02-18T11:48:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.969+0000 7f8e9effd640 1 -- 172.21.15.116:0/2301616469 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90031940 con 0x7f8ea0145230 2024-02-18T11:48:19.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.969+0000 7f8e9effd640 1 -- 172.21.15.116:0/2301616469 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90031cd0 con 0x7f8ea0145230 2024-02-18T11:48:19.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 -- 172.21.15.116:0/2301616469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea0145230 msgr2=0x7f8ea015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2301616469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea0145230 0x7f8ea015be90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ea006f320 tx=0x7f8e9002b880 comp rx=0 tx=0).stop 2024-02-18T11:48:19.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 -- 172.21.15.116:0/2301616469 shutdown_connections 2024-02-18T11:48:19.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2301616469 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea015c3d0 0x7f8ea00711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.966 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2301616469 >> v2:172.21.15.143:3300/0 conn(0x7f8ea0071700 0x7f8ea006d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2301616469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea0145230 0x7f8ea015be90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 -- 172.21.15.116:0/2301616469 >> 172.21.15.116:0/2301616469 conn(0x7f8ea0067000 msgr2=0x7f8ea006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 -- 172.21.15.116:0/2301616469 shutdown_connections 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.970+0000 7f8ea6327640 1 -- 172.21.15.116:0/2301616469 wait complete. 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.971+0000 7f8ea6327640 1 Processor -- start 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.971+0000 7f8ea6327640 1 -- start start 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea6327640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea0071700 0x7f8ea00c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea6327640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ea0145230 0x7f8ea00adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea6327640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 0x7f8ea00ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea6327640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea00c6b20 con 0x7f8ea015c3d0 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea6327640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea00b26f0 con 0x7f8ea0145230 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea6327640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea01f7ec0 con 0x7f8ea0071700 2024-02-18T11:48:19.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 0x7f8ea00ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 0x7f8ea00ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44758/0 (socket says 172.21.15.116:44758) 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 -- 172.21.15.116:0/2254138855 learned_addr learned my addr 172.21.15.116:0/2254138855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8e9f7fe640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ea0145230 0x7f8ea00adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 -- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea0071700 msgr2=0x7f8ea00c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea0071700 0x7f8ea00c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 -- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ea0145230 msgr2=0x7f8ea00adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ea0145230 0x7f8ea00adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 -- 172.21.15.116:0/2254138855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e900094d0 con 0x7f8ea015c3d0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8ea48c4640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 0x7f8ea00ae030 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e98007e10 tx=0x7f8e9800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.972+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98003b90 con 0x7f8ea015c3d0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.973+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800f040 con 0x7f8ea015c3d0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.973+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800de80 con 0x7f8ea015c3d0 2024-02-18T11:48:19.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.973+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea00b2990 con 0x7f8ea015c3d0 2024-02-18T11:48:19.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.973+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea00b2f50 con 0x7f8ea015c3d0 2024-02-18T11:48:19.969 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.974+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea006baf0 con 0x7f8ea015c3d0 2024-02-18T11:48:19.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.974+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f8e98004030 con 0x7f8ea015c3d0 2024-02-18T11:48:19.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.974+0000 7f8e9cff9640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e8c06bc70 0x7f8e8c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:19.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.975+0000 7f8e9ffff640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e8c06bc70 0x7f8e8c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:19.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.975+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f8e9808ae60 con 0x7f8ea015c3d0 2024-02-18T11:48:19.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.975+0000 7f8e9ffff640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e8c06bc70 0x7f8e8c06e130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e90036000 tx=0x7f8e90030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:19.972 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:19.979+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e980552b0 con 0x7f8ea015c3d0 2024-02-18T11:48:20.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.150+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ea00b3140 con 0x7f8ea015c3d0 2024-02-18T11:48:20.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.150+0000 7f8e9cff9640 1 -- 172.21.15.116:0/2254138855 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e9801a0a0 con 0x7f8ea015c3d0 2024-02-18T11:48:20.144 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e8c06bc70 msgr2=0x7f8e8c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e8c06bc70 0x7f8e8c06e130 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e90036000 tx=0x7f8e90030000 comp rx=0 tx=0).stop 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 msgr2=0x7f8ea00ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 0x7f8ea00ae030 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e98007e10 tx=0x7f8e9800d4a0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 shutdown_connections 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f8e8c06bc70 0x7f8e8c06e130 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea015c3d0 0x7f8ea00ae030 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8ea0145230 0x7f8ea00adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 --2- 172.21.15.116:0/2254138855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8ea0071700 0x7f8ea00c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 >> 172.21.15.116:0/2254138855 conn(0x7f8ea0067000 msgr2=0x7f8ea015e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:20.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 shutdown_connections 2024-02-18T11:48:20.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.152+0000 7f8ea6327640 1 -- 172.21.15.116:0/2254138855 wait complete. 2024-02-18T11:48:20.156 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:20.258 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.265+0000 7feb242d4640 1 Processor -- start 2024-02-18T11:48:20.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.265+0000 7feb242d4640 1 -- start start 2024-02-18T11:48:20.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb242d4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feb1c15dbf0 0x7feb1c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb242d4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb242d4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb1c0d1ee0 con 0x7feb1c0b9370 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb242d4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb1c161dc0 con 0x7feb1c15dbf0 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb242d4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1c1620a0 con 0x7feb1c0cee50 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb22070640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feb1c15dbf0 0x7feb1c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb2186f640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7feb1c0b9370 0x7feb1c0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:43606/0 (socket says 172.21.15.116:43606) 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb22871640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.266+0000 7feb2186f640 1 -- 172.21.15.116:0/1354113796 learned_addr learned my addr 172.21.15.116:0/1354113796 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.267+0000 7feb22070640 1 -- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 msgr2=0x7feb1c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.267+0000 7feb22070640 1 --2- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.267+0000 7feb22070640 1 -- 172.21.15.116:0/1354113796 >> v1:172.21.15.143:6789/0 conn(0x7feb1c0b9370 legacy=0x7feb1c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.267+0000 7feb22070640 1 -- 172.21.15.116:0/1354113796 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c162380 con 0x7feb1c15dbf0 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.267+0000 7feb2106e640 1 -- 172.21.15.116:0/1354113796 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136116065 0 0) 0x7feb1c0d1ee0 con 0x7feb1c0b9370 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.267+0000 7feb22871640 1 --2- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb22070640 1 --2- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c1617e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009d90 tx=0x7feb0c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f7aaac5df15d7e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb2106e640 1 -- 172.21.15.116:0/1354113796 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c02cc50 con 0x7feb1c15dbf0 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb2106e640 1 -- 172.21.15.116:0/1354113796 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c02cdb0 con 0x7feb1c15dbf0 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb2106e640 1 -- 172.21.15.116:0/1354113796 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c034660 con 0x7feb1c15dbf0 2024-02-18T11:48:20.262 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 -- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 msgr2=0x7feb1c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 --2- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c1617e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009d90 tx=0x7feb0c02bea0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 -- 172.21.15.116:0/1354113796 shutdown_connections 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 --2- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 --2- 172.21.15.116:0/1354113796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c1617e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 -- 172.21.15.116:0/1354113796 >> 172.21.15.116:0/1354113796 conn(0x7feb1c0b5480 msgr2=0x7feb1c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 -- 172.21.15.116:0/1354113796 shutdown_connections 2024-02-18T11:48:20.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.268+0000 7feb242d4640 1 -- 172.21.15.116:0/1354113796 wait complete. 2024-02-18T11:48:20.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 Processor -- start 2024-02-18T11:48:20.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 -- start start 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 0x7feb1c15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1c161150 con 0x7feb1c0b9370 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1c15fb70 con 0x7feb1c15dbf0 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb242d4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1c15fe50 con 0x7feb1c0cee50 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb22070640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 0x7feb1c15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb22070640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 0x7feb1c15f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44766/0 (socket says 172.21.15.116:44766) 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.270+0000 7feb22070640 1 -- 172.21.15.116:0/1201015179 learned_addr learned my addr 172.21.15.116:0/1201015179 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb2186f640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22871640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22070640 1 -- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 msgr2=0x7feb1c160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22070640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22070640 1 -- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 msgr2=0x7feb1c14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22070640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c14ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22070640 1 -- 172.21.15.116:0/1201015179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14009400 con 0x7feb1c0b9370 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22871640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb2186f640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb22070640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 0x7feb1c15f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009d90 tx=0x7feb0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c040450 con 0x7feb1c0b9370 2024-02-18T11:48:20.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb0c0405b0 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0409a0 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c009a70 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.271+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1c15e150 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.272+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1c067840 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.272+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7feb0c040b00 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.273+0000 7feb1affd640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7feb0806bca0 0x7feb0806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.273+0000 7feb2186f640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7feb0806bca0 0x7feb0806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.273+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7feb0c0ad600 con 0x7feb1c0b9370 2024-02-18T11:48:20.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.273+0000 7feb2186f640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7feb0806bca0 0x7feb0806e160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feb1c1589f0 tx=0x7feb10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.277+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb0c077b00 con 0x7feb1c0b9370 2024-02-18T11:48:20.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.447+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb1c074520 con 0x7feb1c0b9370 2024-02-18T11:48:20.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.448+0000 7feb1affd640 1 -- 172.21.15.116:0/1201015179 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb0c040db0 con 0x7feb1c0b9370 2024-02-18T11:48:20.441 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:20.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7feb0806bca0 msgr2=0x7feb0806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7feb0806bca0 0x7feb0806e160 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7feb1c1589f0 tx=0x7feb10009040 comp rx=0 tx=0).stop 2024-02-18T11:48:20.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 msgr2=0x7feb1c15f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 0x7feb1c15f590 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feb0c009d90 tx=0x7feb0c004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:20.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 shutdown_connections 2024-02-18T11:48:20.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7feb0806bca0 0x7feb0806e160 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb1c15dbf0 0x7feb1c14ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb1c0cee50 0x7feb1c160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 --2- 172.21.15.116:0/1201015179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb1c0b9370 0x7feb1c15f590 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 >> 172.21.15.116:0/1201015179 conn(0x7feb1c0b5480 msgr2=0x7feb1c06c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 shutdown_connections 2024-02-18T11:48:20.444 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.450+0000 7feb242d4640 1 -- 172.21.15.116:0/1201015179 wait complete. 2024-02-18T11:48:20.454 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:20.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.565+0000 7ff5fc22e640 1 Processor -- start 2024-02-18T11:48:20.558 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.565+0000 7ff5fc22e640 1 -- start start 2024-02-18T11:48:20.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fc22e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f415cb20 0x7ff5f40cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.568 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fc22e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5f40cfeb0 0x7ff5f406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fc22e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40d0320 con 0x7ff5f40afc00 2024-02-18T11:48:20.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fc22e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f40703c0 con 0x7ff5f40cfeb0 2024-02-18T11:48:20.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fc22e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f40cd820 con 0x7ff5f415cb20 2024-02-18T11:48:20.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fa7cb640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5f40cfeb0 0x7ff5f406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5f97c9640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f415cb20 0x7ff5f40cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.569 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5f9fca640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff5f40afc00 0x7ff5f406fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:43618/0 (socket says 172.21.15.116:43618) 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5f9fca640 1 -- 172.21.15.116:0/3534774277 learned_addr learned my addr 172.21.15.116:0/3534774277 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f415cb20 msgr2=0x7ff5f40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fa7cb640 1 --2- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f415cb20 0x7ff5f40cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/3534774277 >> v1:172.21.15.143:6789/0 conn(0x7ff5f40afc00 legacy=0x7ff5f406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/3534774277 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40affe0 con 0x7ff5f40cfeb0 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5f8fc8640 1 -- 172.21.15.116:0/3534774277 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534210129 0 0) 0x7ff5f0001110 con 0x7ff5f40afc00 2024-02-18T11:48:20.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.566+0000 7ff5f97c9640 1 --2- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f415cb20 0x7ff5f40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:20.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.567+0000 7ff5fa7cb640 1 --2- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f406e370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009d90 tx=0x7ff5ec02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a1af2a896009a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.567+0000 7ff5f8fc8640 1 -- 172.21.15.116:0/3534774277 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec02cbc0 con 0x7ff5f40cfeb0 2024-02-18T11:48:20.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.567+0000 7ff5f8fc8640 1 -- 172.21.15.116:0/3534774277 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec02cd20 con 0x7ff5f40cfeb0 2024-02-18T11:48:20.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.567+0000 7ff5f8fc8640 1 -- 172.21.15.116:0/3534774277 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec034660 con 0x7ff5f40cfeb0 2024-02-18T11:48:20.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 -- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 msgr2=0x7ff5f406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f406e370 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009d90 tx=0x7ff5ec02be10 comp rx=0 tx=0).stop 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 -- 172.21.15.116:0/3534774277 shutdown_connections 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f406e370 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/3534774277 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f415cb20 0x7ff5f40cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 -- 172.21.15.116:0/3534774277 >> 172.21.15.116:0/3534774277 conn(0x7ff5f40b2230 msgr2=0x7ff5f40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 -- 172.21.15.116:0/3534774277 shutdown_connections 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.568+0000 7ff5fc22e640 1 -- 172.21.15.116:0/3534774277 wait complete. 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.569+0000 7ff5fc22e640 1 Processor -- start 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.569+0000 7ff5fc22e640 1 -- start start 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fc22e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f40afc00 0x7ff5f414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fc22e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f40b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fc22e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 0x7ff5f40b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fc22e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f40ae100 con 0x7ff5f415cb20 2024-02-18T11:48:20.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fc22e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f40ca4c0 con 0x7ff5f40cfeb0 2024-02-18T11:48:20.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fc22e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f40ad510 con 0x7ff5f40afc00 2024-02-18T11:48:20.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 0x7ff5f40b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 0x7ff5f40b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44768/0 (socket says 172.21.15.116:44768) 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/2512691967 learned_addr learned my addr 172.21.15.116:0/2512691967 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5f9fca640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f40afc00 0x7ff5f414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5f97c9640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f40b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f40afc00 msgr2=0x7ff5f414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f40afc00 0x7ff5f414f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 msgr2=0x7ff5f40b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f40b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 -- 172.21.15.116:0/2512691967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4009400 con 0x7ff5f415cb20 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.570+0000 7ff5fa7cb640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 0x7ff5f40b3ee0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec037040 tx=0x7ff5ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.571+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec003b90 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.571+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec02ce90 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.571+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec009a70 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.571+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec009450 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.571+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f40ad760 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.571+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f40cdc90 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.572+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff5ec004030 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.572+0000 7ff5ea7fc640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5d806bc70 0x7ff5d806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.572+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7ff5ec0ad1c0 con 0x7ff5f415cb20 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.572+0000 7ff5f9fca640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5d806bc70 0x7ff5d806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.573+0000 7ff5f9fca640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5d806bc70 0x7ff5d806e130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0007fd0 tx=0x7ff5f0007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.577+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5f40cdc90 con 0x7ff5f415cb20 2024-02-18T11:48:20.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.748+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5f40cdc90 con 0x7ff5f415cb20 2024-02-18T11:48:20.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.748+0000 7ff5ea7fc640 1 -- 172.21.15.116:0/2512691967 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5f40cdc90 con 0x7ff5f415cb20 2024-02-18T11:48:20.742 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5d806bc70 msgr2=0x7ff5d806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5d806bc70 0x7ff5d806e130 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0007fd0 tx=0x7ff5f0007de0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 msgr2=0x7ff5f40b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 0x7ff5f40b3ee0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec037040 tx=0x7ff5ec004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 shutdown_connections 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff5d806bc70 0x7ff5d806e130 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f415cb20 0x7ff5f40b3ee0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5f40cfeb0 0x7ff5f40b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 --2- 172.21.15.116:0/2512691967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5f40afc00 0x7ff5f414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 >> 172.21.15.116:0/2512691967 conn(0x7ff5f40b2230 msgr2=0x7ff5f40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:20.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 shutdown_connections 2024-02-18T11:48:20.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.750+0000 7ff5fc22e640 1 -- 172.21.15.116:0/2512691967 wait complete. 2024-02-18T11:48:20.754 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:20.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.864+0000 7f6a5d8df640 1 Processor -- start 2024-02-18T11:48:20.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.864+0000 7f6a5d8df640 1 -- start start 2024-02-18T11:48:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.864+0000 7f6a5d8df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a5815dbf0 0x7f6a581617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.864+0000 7f6a5d8df640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a580d1ee0 con 0x7f6a580cee50 2024-02-18T11:48:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.864+0000 7f6a5d8df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a58161dc0 con 0x7f6a580b9370 2024-02-18T11:48:20.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.864+0000 7f6a5d8df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a581620a0 con 0x7f6a5815dbf0 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a577fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6a580cee50 0x7f6a580d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53660/0 (socket says 172.21.15.116:53660) 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a577fe640 1 -- 172.21.15.116:0/84094719 learned_addr learned my addr 172.21.15.116:0/84094719 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a56ffd640 1 --2- 172.21.15.116:0/84094719 >> v2:172.21.15.143:3300/0 conn(0x7f6a5815dbf0 0x7f6a581617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465830608 0 0) 0x7f6a580d1ee0 con 0x7f6a580cee50 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a40003520 con 0x7f6a580cee50 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445761444 0 0) 0x7f6a40003520 con 0x7f6a580cee50 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 >> v2:172.21.15.143:3300/0 conn(0x7f6a5815dbf0 msgr2=0x7f6a581617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 --2- 172.21.15.116:0/84094719 >> v2:172.21.15.143:3300/0 conn(0x7f6a5815dbf0 0x7f6a581617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 >> v1:172.21.15.143:6789/0 conn(0x7f6a580b9370 legacy=0x7f6a580ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58162380 con 0x7f6a580cee50 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f6a500019f0 con 0x7f6a580cee50 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 >> v1:172.21.15.116:6789/0 conn(0x7f6a580cee50 legacy=0x7f6a580d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a40003970 0x7f6a40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a581620a0 con 0x7f6a400047c0 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4000c020 con 0x7f6a40003970 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4000c1e0 con 0x7f6a40005180 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a567fc640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a56ffd640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a40003970 0x7f6a40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a577fe640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a567fc640 1 -- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a40005180 msgr2=0x7f6a4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a567fc640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a40005180 0x7f6a4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.865+0000 7f6a567fc640 1 -- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a40003970 msgr2=0x7f6a40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a567fc640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a40003970 0x7f6a40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.863 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a577fe640 1 -- 172.21.15.116:0/84094719 reap_dead start 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a567fc640 1 -- 172.21.15.116:0/84094719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40003520 con 0x7f6a400047c0 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a567fc640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c001750 tx=0x7f6a400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a516fe34a3394 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0052d0 con 0x7f6a400047c0 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c02e050 con 0x7f6a400047c0 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.866+0000 7f6a55ffb640 1 -- 172.21.15.116:0/84094719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c003830 con 0x7f6a400047c0 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 -- 172.21.15.116:0/84094719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 msgr2=0x7f6a40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c001750 tx=0x7f6a400057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 -- 172.21.15.116:0/84094719 shutdown_connections 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/84094719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a40004c40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 -- 172.21.15.116:0/84094719 >> 172.21.15.116:0/84094719 conn(0x7f6a580b5480 msgr2=0x7f6a580b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 -- 172.21.15.116:0/84094719 shutdown_connections 2024-02-18T11:48:20.864 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.867+0000 7f6a5d8df640 1 -- 172.21.15.116:0/84094719 wait complete. 2024-02-18T11:48:20.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.868+0000 7f6a5d8df640 1 Processor -- start 2024-02-18T11:48:20.865 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.868+0000 7f6a5d8df640 1 -- start start 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a5d8df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a5815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a5d8df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5814ea00 0x7f6a58151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a5d8df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a58152060 0x7f6a580cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a5d8df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5815acc0 con 0x7f6a400047c0 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a5d8df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5815b690 con 0x7f6a5814ea00 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a5d8df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5815a0b0 con 0x7f6a58152060 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a5815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a5815b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44782/0 (socket says 172.21.15.116:44782) 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 -- 172.21.15.116:0/1527427488 learned_addr learned my addr 172.21.15.116:0/1527427488 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a577fe640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a58152060 0x7f6a580cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.868 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a567fc640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5814ea00 0x7f6a58151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 -- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a58152060 msgr2=0x7f6a580cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a58152060 0x7f6a580cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 -- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5814ea00 msgr2=0x7f6a58151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5814ea00 0x7f6a58151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 -- 172.21.15.116:0/1527427488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5815e150 con 0x7f6a400047c0 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a577fe640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a58152060 0x7f6a580cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a567fc640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5814ea00 0x7f6a58151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.869+0000 7f6a56ffd640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a5815b150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a480118c0 tx=0x7f6a48011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.870+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48004240 con 0x7f6a400047c0 2024-02-18T11:48:20.869 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.870+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a480043a0 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.870+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5815a390 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.870+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800fa20 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.870+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5815a950 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.871+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58067840 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.871+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f6a4800fb80 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.871+0000 7f6a3ffff640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6a4406bc70 0x7f6a4406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.872+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6a4808b220 con 0x7f6a400047c0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.872+0000 7f6a567fc640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6a4406bc70 0x7f6a4406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.872+0000 7f6a567fc640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6a4406bc70 0x7f6a4406e130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c033000 tx=0x7f6a4c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:20.870 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:20.876+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a480555e0 con 0x7f6a400047c0 2024-02-18T11:48:21.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.045+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6a58074520 con 0x7f6a400047c0 2024-02-18T11:48:21.039 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.045+0000 7f6a3ffff640 1 -- 172.21.15.116:0/1527427488 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6a48019020 con 0x7f6a400047c0 2024-02-18T11:48:21.039 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6a4406bc70 msgr2=0x7f6a4406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6a4406bc70 0x7f6a4406e130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c033000 tx=0x7f6a4c009030 comp rx=0 tx=0).stop 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 msgr2=0x7f6a5815b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a5815b150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6a480118c0 tx=0x7f6a48011d80 comp rx=0 tx=0).stop 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 shutdown_connections 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f6a4406bc70 0x7f6a4406e130 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6a58152060 0x7f6a580cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a5814ea00 0x7f6a58151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 --2- 172.21.15.116:0/1527427488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a400047c0 0x7f6a5815b150 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 >> 172.21.15.116:0/1527427488 conn(0x7f6a580b5480 msgr2=0x7f6a580d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 shutdown_connections 2024-02-18T11:48:21.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.047+0000 7f6a5d8df640 1 -- 172.21.15.116:0/1527427488 wait complete. 2024-02-18T11:48:21.078 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-02-18T11:48:21.079 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:21.179 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.186+0000 7ff577aec640 1 Processor -- start 2024-02-18T11:48:21.180 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.186+0000 7ff577aec640 1 -- start start 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff577aec640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff570160e90 0x7ff5700afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff577aec640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff57006aff0 0x7ff570070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff577aec640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57006b480 con 0x7ff57006ab60 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff577aec640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5700d14a0 con 0x7ff57006aff0 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff577aec640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5700d0400 con 0x7ff570160e90 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff576089640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff57006aff0 0x7ff570070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff576089640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff57006aff0 0x7ff570070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44792/0 (socket says 172.21.15.116:44792) 2024-02-18T11:48:21.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.187+0000 7ff576089640 1 -- 172.21.15.116:0/1330996609 learned_addr learned my addr 172.21.15.116:0/1330996609 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:21.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff576089640 1 -- 172.21.15.116:0/1330996609 >> v2:172.21.15.143:3300/0 conn(0x7ff570160e90 msgr2=0x7ff5700afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:21.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff576089640 1 --2- 172.21.15.116:0/1330996609 >> v2:172.21.15.143:3300/0 conn(0x7ff570160e90 0x7ff5700afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff576089640 1 -- 172.21.15.116:0/1330996609 >> v1:172.21.15.143:6789/0 conn(0x7ff57006ab60 legacy=0x7ff57006fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:21.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff576089640 1 -- 172.21.15.116:0/1330996609 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700d06e0 con 0x7ff57006aff0 2024-02-18T11:48:21.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff576089640 1 --2- 172.21.15.116:0/1330996609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff57006aff0 0x7ff570070180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff56802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fbf1dd919baf89 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff574886640 1 -- 172.21.15.116:0/1330996609 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568031870 con 0x7ff57006aff0 2024-02-18T11:48:21.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff574886640 1 -- 172.21.15.116:0/1330996609 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5680319d0 con 0x7ff57006aff0 2024-02-18T11:48:21.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.188+0000 7ff574886640 1 -- 172.21.15.116:0/1330996609 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568031d60 con 0x7ff57006aff0 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 -- 172.21.15.116:0/1330996609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff57006aff0 msgr2=0x7ff570070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 --2- 172.21.15.116:0/1330996609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff57006aff0 0x7ff570070180 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff56802b910 comp rx=0 tx=0).stop 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 -- 172.21.15.116:0/1330996609 shutdown_connections 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 --2- 172.21.15.116:0/1330996609 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff57006aff0 0x7ff570070180 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 --2- 172.21.15.116:0/1330996609 >> v2:172.21.15.143:3300/0 conn(0x7ff570160e90 0x7ff5700afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 -- 172.21.15.116:0/1330996609 >> 172.21.15.116:0/1330996609 conn(0x7ff5700b23f0 msgr2=0x7ff5700b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 -- 172.21.15.116:0/1330996609 shutdown_connections 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.189+0000 7ff577aec640 1 -- 172.21.15.116:0/1330996609 wait complete. 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.190+0000 7ff577aec640 1 Processor -- start 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.190+0000 7ff577aec640 1 -- start start 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff577aec640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57006ab60 0x7ff57014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff577aec640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff57006aff0 0x7ff570157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff577aec640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 0x7ff57015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff577aec640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5701575a0 con 0x7ff570160e90 2024-02-18T11:48:21.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff577aec640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700cc260 con 0x7ff57006ab60 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff577aec640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff570157ec0 con 0x7ff57006aff0 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 0x7ff57015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 0x7ff57015c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44804/0 (socket says 172.21.15.116:44804) 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 -- 172.21.15.116:0/2159224275 learned_addr learned my addr 172.21.15.116:0/2159224275 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff575888640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57006ab60 0x7ff57014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff575087640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff57006aff0 0x7ff570157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 -- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff57006aff0 msgr2=0x7ff570157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff57006aff0 0x7ff570157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 -- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57006ab60 msgr2=0x7ff57014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57006ab60 0x7ff57014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 -- 172.21.15.116:0/2159224275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680094d0 con 0x7ff570160e90 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.191+0000 7ff576089640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 0x7ff57015c700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff56802b950 tx=0x7ff5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.192+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56803adf0 con 0x7ff570160e90 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.192+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5680443f0 con 0x7ff570160e90 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.192+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5700cb650 con 0x7ff570160e90 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.192+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680447a0 con 0x7ff570160e90 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.192+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5700cbc10 con 0x7ff570160e90 2024-02-18T11:48:21.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.193+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5700cec00 con 0x7ff570160e90 2024-02-18T11:48:21.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.193+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff568044900 con 0x7ff570160e90 2024-02-18T11:48:21.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.193+0000 7ff5667fc640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff55406bc70 0x7ff55406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.194+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7ff56802d040 con 0x7ff570160e90 2024-02-18T11:48:21.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.194+0000 7ff575888640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff55406bc70 0x7ff55406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.194+0000 7ff575888640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff55406bc70 0x7ff55406e130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f20 tx=0x7ff56c0094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.191 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.198+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff568077730 con 0x7ff570160e90 2024-02-18T11:48:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.369+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff570074520 con 0x7ff570160e90 2024-02-18T11:48:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.370+0000 7ff5667fc640 1 -- 172.21.15.116:0/2159224275 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7ff56807adf0 con 0x7ff570160e90 2024-02-18T11:48:21.364 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:21.364 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff55406bc70 msgr2=0x7ff55406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff55406bc70 0x7ff55406e130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009f20 tx=0x7ff56c0094f0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 msgr2=0x7ff57015c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 0x7ff57015c700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff56802b950 tx=0x7ff5680046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 shutdown_connections 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff55406bc70 0x7ff55406e130 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff570160e90 0x7ff57015c700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff57006aff0 0x7ff570157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 --2- 172.21.15.116:0/2159224275 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff57006ab60 0x7ff57014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 >> 172.21.15.116:0/2159224275 conn(0x7ff5700b23f0 msgr2=0x7ff5700cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 shutdown_connections 2024-02-18T11:48:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.372+0000 7ff577aec640 1 -- 172.21.15.116:0/2159224275 wait complete. 2024-02-18T11:48:21.377 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:21.377 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:21.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.486+0000 7fc9161e2640 1 Processor -- start 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.486+0000 7fc9161e2640 1 -- start start 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc9161e2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc910071a50 0x7fc910071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc9161e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9100cf0e0 0x7fc91006b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc9161e2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc910072370 con 0x7fc9100726c0 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc9161e2640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9100cdc80 con 0x7fc910071a50 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc9161e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91015c500 con 0x7fc9100cf0e0 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc90effd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc9100726c0 0x7fc91006ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53676/0 (socket says 172.21.15.116:53676) 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc90effd640 1 -- 172.21.15.116:0/3409041839 learned_addr learned my addr 172.21.15.116:0/3409041839 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/3409041839 >> v2:172.21.15.143:3300/0 conn(0x7fc910071a50 0x7fc910071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc90ffff640 1 --2- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9100cf0e0 0x7fc91006b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc90e7fc640 1 -- 172.21.15.116:0/3409041839 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593202944 0 0) 0x7fc910072370 con 0x7fc9100726c0 2024-02-18T11:48:21.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.487+0000 7fc90e7fc640 1 -- 172.21.15.116:0/3409041839 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f8003520 con 0x7fc9100726c0 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.488+0000 7fc90e7fc640 1 -- 172.21.15.116:0/3409041839 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328799165 0 0) 0x7fc8f8003520 con 0x7fc9100726c0 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.488+0000 7fc90f7fe640 1 -- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9100cf0e0 msgr2=0x7fc91006b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.488+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9100cf0e0 0x7fc91006b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.488+0000 7fc90f7fe640 1 -- 172.21.15.116:0/3409041839 >> v1:172.21.15.116:6789/0 conn(0x7fc9100726c0 legacy=0x7fc91006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.488+0000 7fc90f7fe640 1 -- 172.21.15.116:0/3409041839 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91015c7e0 con 0x7fc910071a50 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.488+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc910071a50 0x7fc910071e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097a0 tx=0x7fc90002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c994cdff6a79a35 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc90e7fc640 1 -- 172.21.15.116:0/3409041839 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904001670 con 0x7fc910071a50 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc90e7fc640 1 -- 172.21.15.116:0/3409041839 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90002e050 con 0x7fc910071a50 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc90e7fc640 1 -- 172.21.15.116:0/3409041839 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900030710 con 0x7fc910071a50 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 -- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc910071a50 msgr2=0x7fc910071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc910071a50 0x7fc910071e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097a0 tx=0x7fc90002b790 comp rx=0 tx=0).stop 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 -- 172.21.15.116:0/3409041839 shutdown_connections 2024-02-18T11:48:21.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9100cf0e0 0x7fc91006b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3409041839 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc910071a50 0x7fc910071e30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 -- 172.21.15.116:0/3409041839 >> 172.21.15.116:0/3409041839 conn(0x7fc9100cfd80 msgr2=0x7fc9100d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 -- 172.21.15.116:0/3409041839 shutdown_connections 2024-02-18T11:48:21.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.489+0000 7fc9161e2640 1 -- 172.21.15.116:0/3409041839 wait complete. 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.490+0000 7fc9161e2640 1 Processor -- start 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 -- start start 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc910071a50 0x7fc9101456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 0x7fc91015a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc91015abd0 0x7fc910145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91015b080 con 0x7fc9100cf0e0 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91009cab0 con 0x7fc91015abd0 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc9161e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc910146ab0 con 0x7fc910071a50 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc90effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 0x7fc91015a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc90effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 0x7fc91015a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44820/0 (socket says 172.21.15.116:44820) 2024-02-18T11:48:21.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.491+0000 7fc90effd640 1 -- 172.21.15.116:0/3865177994 learned_addr learned my addr 172.21.15.116:0/3865177994 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc910071a50 0x7fc9101456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90ffff640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc91015abd0 0x7fc910145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90effd640 1 -- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc910071a50 msgr2=0x7fc9101456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90effd640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc910071a50 0x7fc9101456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90effd640 1 -- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc91015abd0 msgr2=0x7fc910145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90effd640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc91015abd0 0x7fc910145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90effd640 1 -- 172.21.15.116:0/3865177994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009400 con 0x7fc9100cf0e0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc90effd640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 0x7fc91015a690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c780 tx=0x7fc90400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904005ad0 con 0x7fc9100cf0e0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904005c50 con 0x7fc9100cf0e0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc910146d90 con 0x7fc9100cf0e0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904007070 con 0x7fc9100cf0e0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.492+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc910147350 con 0x7fc9100cf0e0 2024-02-18T11:48:21.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.493+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc910067180 con 0x7fc9100cf0e0 2024-02-18T11:48:21.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.494+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fc9040071d0 con 0x7fc9100cf0e0 2024-02-18T11:48:21.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.494+0000 7fc8f7fff640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc8fc06bc70 0x7fc8fc06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.494+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc8fc06bc70 0x7fc8fc06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.494+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc90408a7a0 con 0x7fc9100cf0e0 2024-02-18T11:48:21.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.494+0000 7fc90f7fe640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc8fc06bc70 0x7fc8fc06e130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc90002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.498+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc904054c20 con 0x7fc9100cf0e0 2024-02-18T11:48:21.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.670+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc910067770 con 0x7fc9100cf0e0 2024-02-18T11:48:21.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.671+0000 7fc8f7fff640 1 -- 172.21.15.116:0/3865177994 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9040582e0 con 0x7fc9100cf0e0 2024-02-18T11:48:21.664 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc8fc06bc70 msgr2=0x7fc8fc06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc8fc06bc70 0x7fc8fc06e130 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc90002f000 comp rx=0 tx=0).stop 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 msgr2=0x7fc91015a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 0x7fc91015a690 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c780 tx=0x7fc90400cc40 comp rx=0 tx=0).stop 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 shutdown_connections 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fc8fc06bc70 0x7fc8fc06e130 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc91015abd0 0x7fc910145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9100cf0e0 0x7fc91015a690 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 --2- 172.21.15.116:0/3865177994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc910071a50 0x7fc9101456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.672+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 >> 172.21.15.116:0/3865177994 conn(0x7fc9100cfd80 msgr2=0x7fc9100b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.673+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 shutdown_connections 2024-02-18T11:48:21.667 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.673+0000 7fc9161e2640 1 -- 172.21.15.116:0/3865177994 wait complete. 2024-02-18T11:48:21.677 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:21.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.786+0000 7f5d9d4ae640 1 Processor -- start 2024-02-18T11:48:21.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.786+0000 7f5d9d4ae640 1 -- start start 2024-02-18T11:48:21.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d9d4ae640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d9815dbf0 0x7f5d981617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d9d4ae640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5d980b9370 0x7f5d980ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d9d4ae640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d980cec80 0x7f5d9806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d9d4ae640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d980cf110 con 0x7f5d980cec80 2024-02-18T11:48:21.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d9d4ae640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d980d1ee0 con 0x7f5d980b9370 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d9d4ae640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98161dc0 con 0x7f5d9815dbf0 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d980cec80 0x7f5d9806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d980cec80 0x7f5d9806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44822/0 (socket says 172.21.15.116:44822) 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 -- 172.21.15.116:0/3699540730 learned_addr learned my addr 172.21.15.116:0/3699540730 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d967fc640 1 --2- 172.21.15.116:0/3699540730 >> v2:172.21.15.143:3300/0 conn(0x7f5d980b9370 0x7f5d980ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d96ffd640 1 --2- 172.21.15.116:0/3699540730 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d9815dbf0 0x7f5d981617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 -- 172.21.15.116:0/3699540730 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d9815dbf0 msgr2=0x7f5d981617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 --2- 172.21.15.116:0/3699540730 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d9815dbf0 0x7f5d981617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 -- 172.21.15.116:0/3699540730 >> v2:172.21.15.143:3300/0 conn(0x7f5d980b9370 msgr2=0x7f5d980ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 --2- 172.21.15.116:0/3699540730 >> v2:172.21.15.143:3300/0 conn(0x7f5d980b9370 0x7f5d980ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.787+0000 7f5d977fe640 1 -- 172.21.15.116:0/3699540730 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d981620a0 con 0x7f5d980cec80 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.788+0000 7f5d977fe640 1 --2- 172.21.15.116:0/3699540730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d9806c210 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5d98068310 tx=0x7f5d9002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c800c46b34bee592 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.788+0000 7f5d95ffb640 1 -- 172.21.15.116:0/3699540730 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9002ca70 con 0x7f5d980cec80 2024-02-18T11:48:21.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.788+0000 7f5d95ffb640 1 -- 172.21.15.116:0/3699540730 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9002cbd0 con 0x7f5d980cec80 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.788+0000 7f5d95ffb640 1 -- 172.21.15.116:0/3699540730 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900343f0 con 0x7f5d980cec80 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/3699540730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 msgr2=0x7f5d9806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/3699540730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d9806c210 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5d98068310 tx=0x7f5d9002bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/3699540730 shutdown_connections 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/3699540730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d9806c210 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/3699540730 >> v2:172.21.15.143:3300/0 conn(0x7f5d980b9370 0x7f5d980ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/3699540730 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d9815dbf0 0x7f5d981617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/3699540730 >> 172.21.15.116:0/3699540730 conn(0x7f5d980b5480 msgr2=0x7f5d980b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/3699540730 shutdown_connections 2024-02-18T11:48:21.784 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.789+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/3699540730 wait complete. 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.790+0000 7f5d9d4ae640 1 Processor -- start 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.790+0000 7f5d9d4ae640 1 -- start start 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d9d4ae640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d980b9370 0x7f5d98151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d9d4ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d98157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d9d4ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9815dbf0 0x7f5d9815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d9d4ae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981580f0 con 0x7f5d980cec80 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d9d4ae640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980c89e0 con 0x7f5d9815dbf0 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d9d4ae640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98157b30 con 0x7f5d980b9370 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d98157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d98157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44834/0 (socket says 172.21.15.116:44834) 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 -- 172.21.15.116:0/4195409709 learned_addr learned my addr 172.21.15.116:0/4195409709 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d96ffd640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d980b9370 0x7f5d98151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d977fe640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9815dbf0 0x7f5d9815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 -- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d980b9370 msgr2=0x7f5d98151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d980b9370 0x7f5d98151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 -- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9815dbf0 msgr2=0x7f5d9815ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9815dbf0 0x7f5d9815ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 -- 172.21.15.116:0/4195409709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90009b30 con 0x7f5d980cec80 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.791+0000 7f5d967fc640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d98157040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d800 tx=0x7f5d8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.792+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0096a0 con 0x7f5d980cec80 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.792+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c009800 con 0x7f5d980cec80 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.792+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c009b90 con 0x7f5d980cec80 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.792+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9815e150 con 0x7f5d980cec80 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.792+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d980c8670 con 0x7f5d980cec80 2024-02-18T11:48:21.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.793+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d98067840 con 0x7f5d980cec80 2024-02-18T11:48:21.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.793+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f5d8c009cf0 con 0x7f5d980cec80 2024-02-18T11:48:21.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.793+0000 7f5d7f7fe640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d8406bc70 0x7f5d8406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:21.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.793+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f5d8c08ae90 con 0x7f5d980cec80 2024-02-18T11:48:21.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.794+0000 7f5d96ffd640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d8406bc70 0x7f5d8406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:21.789 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.794+0000 7f5d96ffd640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d8406bc70 0x7f5d8406e130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d880079a0 tx=0x7f5d88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:21.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.798+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d8c055300 con 0x7f5d980cec80 2024-02-18T11:48:21.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.967+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d980c84d0 con 0x7f5d980cec80 2024-02-18T11:48:21.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.967+0000 7f5d7f7fe640 1 -- 172.21.15.116:0/4195409709 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d980c84d0 con 0x7f5d980cec80 2024-02-18T11:48:21.961 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d8406bc70 msgr2=0x7f5d8406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d8406bc70 0x7f5d8406e130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d880079a0 tx=0x7f5d88008040 comp rx=0 tx=0).stop 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 msgr2=0x7f5d98157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d98157040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d800 tx=0x7f5d8c00dcc0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 shutdown_connections 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d8406bc70 0x7f5d8406e130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9815dbf0 0x7f5d9815ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d980cec80 0x7f5d98157040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 --2- 172.21.15.116:0/4195409709 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d980b9370 0x7f5d98151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 >> 172.21.15.116:0/4195409709 conn(0x7f5d980b5480 msgr2=0x7f5d98158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 shutdown_connections 2024-02-18T11:48:21.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:21.969+0000 7f5d9d4ae640 1 -- 172.21.15.116:0/4195409709 wait complete. 2024-02-18T11:48:21.974 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:22.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.083+0000 7fd206223640 1 Processor -- start 2024-02-18T11:48:22.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.083+0000 7fd206223640 1 -- start start 2024-02-18T11:48:22.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd206223640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd200144090 0x7fd2000b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd206223640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd2000cc9e0 0x7fd20006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd206223640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2000b2a50 con 0x7fd2000cce20 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd206223640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20015b090 con 0x7fd2000cc9e0 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd206223640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20015b370 con 0x7fd200144090 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd2000cc9e0 0x7fd20006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd2000cc9e0 0x7fd20006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44842/0 (socket says 172.21.15.116:44842) 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 -- 172.21.15.116:0/1009919330 learned_addr learned my addr 172.21.15.116:0/1009919330 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1fffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd2000cce20 0x7fd20015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53678/0 (socket says 172.21.15.116:53678) 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1ff7fe640 1 --2- 172.21.15.116:0/1009919330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd200144090 0x7fd2000b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 -- 172.21.15.116:0/1009919330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd200144090 msgr2=0x7fd2000b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 --2- 172.21.15.116:0/1009919330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd200144090 0x7fd2000b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 -- 172.21.15.116:0/1009919330 >> v1:172.21.15.116:6789/0 conn(0x7fd2000cce20 legacy=0x7fd20015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1feffd640 1 -- 172.21.15.116:0/1009919330 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20015b650 con 0x7fd2000cc9e0 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.084+0000 7fd1fe7fc640 1 -- 172.21.15.116:0/1009919330 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902457928 0 0) 0x7fd2000b2a50 con 0x7fd2000cce20 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.085+0000 7fd1feffd640 1 --2- 172.21.15.116:0/1009919330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cc9e0 0x7fd20006e330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400bb10 tx=0x7fd1f402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd75074fc6952eb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.079 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.085+0000 7fd1fe7fc640 1 -- 172.21.15.116:0/1009919330 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f402ee70 con 0x7fd2000cc9e0 2024-02-18T11:48:22.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.085+0000 7fd1fe7fc640 1 -- 172.21.15.116:0/1009919330 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4036440 con 0x7fd2000cc9e0 2024-02-18T11:48:22.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.085+0000 7fd1fe7fc640 1 -- 172.21.15.116:0/1009919330 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f40367d0 con 0x7fd2000cc9e0 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 -- 172.21.15.116:0/1009919330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cc9e0 msgr2=0x7fd20006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 --2- 172.21.15.116:0/1009919330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cc9e0 0x7fd20006e330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400bb10 tx=0x7fd1f402e0c0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 -- 172.21.15.116:0/1009919330 shutdown_connections 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 --2- 172.21.15.116:0/1009919330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cc9e0 0x7fd20006e330 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 --2- 172.21.15.116:0/1009919330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd200144090 0x7fd2000b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 -- 172.21.15.116:0/1009919330 >> 172.21.15.116:0/1009919330 conn(0x7fd200067000 msgr2=0x7fd20006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 -- 172.21.15.116:0/1009919330 shutdown_connections 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.086+0000 7fd206223640 1 -- 172.21.15.116:0/1009919330 wait complete. 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.087+0000 7fd206223640 1 Processor -- start 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.087+0000 7fd206223640 1 -- start start 2024-02-18T11:48:22.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd206223640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2000cc9e0 0x7fd200143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd206223640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 0x7fd2001f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd206223640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd200144090 0x7fd2000b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd206223640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd200141680 con 0x7fd2000cce20 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd206223640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000b4a30 con 0x7fd200144090 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd206223640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000a9a90 con 0x7fd2000cc9e0 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 0x7fd2001f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 0x7fd2001f0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44846/0 (socket says 172.21.15.116:44846) 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 -- 172.21.15.116:0/551141914 learned_addr learned my addr 172.21.15.116:0/551141914 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1ff7fe640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2000cc9e0 0x7fd200143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 -- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2000cc9e0 msgr2=0x7fd200143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2000cc9e0 0x7fd200143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1fffff640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd200144090 0x7fd2000b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 -- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd200144090 msgr2=0x7fd2000b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd200144090 0x7fd2000b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 -- 172.21.15.116:0/551141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f400b770 con 0x7fd2000cce20 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1feffd640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 0x7fd2001f0190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd1f402e100 tx=0x7fd1f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.088+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4041830 con 0x7fd2000cce20 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.089+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f40419b0 con 0x7fd2000cce20 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.089+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4041d60 con 0x7fd2000cce20 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.089+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2000a9d70 con 0x7fd2000cce20 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.089+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2001f06d0 con 0x7fd2000cce20 2024-02-18T11:48:22.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.089+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2000cbe40 con 0x7fd2000cce20 2024-02-18T11:48:22.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.090+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd1f403d040 con 0x7fd2000cce20 2024-02-18T11:48:22.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.090+0000 7fd1e7fff640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd1ec06bc70 0x7fd1ec06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.090+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd1f40af8e0 con 0x7fd2000cce20 2024-02-18T11:48:22.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.090+0000 7fd1ff7fe640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd1ec06bc70 0x7fd1ec06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.091+0000 7fd1ff7fe640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd1ec06bc70 0x7fd1ec06e130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd20006d300 tx=0x7fd1f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.095+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1f4079cb0 con 0x7fd2000cce20 2024-02-18T11:48:22.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.265+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd200074520 con 0x7fd2000cce20 2024-02-18T11:48:22.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.266+0000 7fd1e7fff640 1 -- 172.21.15.116:0/551141914 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1f407d370 con 0x7fd2000cce20 2024-02-18T11:48:22.260 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd1ec06bc70 msgr2=0x7fd1ec06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd1ec06bc70 0x7fd1ec06e130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd20006d300 tx=0x7fd1f0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 msgr2=0x7fd2001f0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 0x7fd2001f0190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd1f402e100 tx=0x7fd1f40046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 shutdown_connections 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd1ec06bc70 0x7fd1ec06e130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd200144090 0x7fd2000b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2000cce20 0x7fd2001f0190 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 --2- 172.21.15.116:0/551141914 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2000cc9e0 0x7fd200143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 >> 172.21.15.116:0/551141914 conn(0x7fd200067000 msgr2=0x7fd20013e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:22.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 shutdown_connections 2024-02-18T11:48:22.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.268+0000 7fd206223640 1 -- 172.21.15.116:0/551141914 wait complete. 2024-02-18T11:48:22.273 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.384+0000 7f512d348640 1 Processor -- start 2024-02-18T11:48:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.384+0000 7f512d348640 1 -- start start 2024-02-18T11:48:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f512d348640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5128145250 0x7f5128070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f512d348640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f51280b5690 0x7f5128071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f512d348640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280ceed0 con 0x7f51280719d0 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f512d348640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280b1100 con 0x7f5128145250 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f512d348640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51280ce2c0 con 0x7f51280b5690 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f5126ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5128145250 0x7f5128070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f51267fc640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f51280719d0 0x7f512815f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53688/0 (socket says 172.21.15.116:53688) 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.385+0000 7f51267fc640 1 -- 172.21.15.116:0/1896578512 learned_addr learned my addr 172.21.15.116:0/1896578512 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f51277fe640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f51280b5690 0x7f5128071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5126ffd640 1 -- 172.21.15.116:0/1896578512 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f51280b5690 msgr2=0x7f5128071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5126ffd640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f51280b5690 0x7f5128071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5126ffd640 1 -- 172.21.15.116:0/1896578512 >> v1:172.21.15.116:6789/0 conn(0x7f51280719d0 legacy=0x7f512815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5125ffb640 1 -- 172.21.15.116:0/1896578512 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848243402 0 0) 0x7f51280ceed0 con 0x7f51280719d0 2024-02-18T11:48:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5126ffd640 1 -- 172.21.15.116:0/1896578512 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280ce5a0 con 0x7f5128145250 2024-02-18T11:48:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f51277fe640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f51280b5690 0x7f5128071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5126ffd640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5128145250 0x7f5128070be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5118009d90 tx=0x7f511802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae33e237e7c4cc11 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.386+0000 7f5125ffb640 1 -- 172.21.15.116:0/1896578512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511802cbc0 con 0x7f5128145250 2024-02-18T11:48:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f5125ffb640 1 -- 172.21.15.116:0/1896578512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511802cd20 con 0x7f5128145250 2024-02-18T11:48:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f5125ffb640 1 -- 172.21.15.116:0/1896578512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118034660 con 0x7f5128145250 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 -- 172.21.15.116:0/1896578512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5128145250 msgr2=0x7f5128070be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5128145250 0x7f5128070be0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5118009d90 tx=0x7f511802be10 comp rx=0 tx=0).stop 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 -- 172.21.15.116:0/1896578512 shutdown_connections 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f51280b5690 0x7f5128071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 --2- 172.21.15.116:0/1896578512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5128145250 0x7f5128070be0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 -- 172.21.15.116:0/1896578512 >> 172.21.15.116:0/1896578512 conn(0x7f512806f460 msgr2=0x7f512806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 -- 172.21.15.116:0/1896578512 shutdown_connections 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.387+0000 7f512d348640 1 -- 172.21.15.116:0/1896578512 wait complete. 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.388+0000 7f512d348640 1 Processor -- start 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.388+0000 7f512d348640 1 -- start start 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f512d348640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51280719d0 0x7f51281583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f512d348640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 0x7f512813e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f512d348640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5128145250 0x7f512813bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f512d348640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51281f11c0 con 0x7f51280b5690 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f512d348640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512813fd00 con 0x7f51280719d0 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f512d348640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512813c590 con 0x7f5128145250 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 0x7f512813e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 0x7f512813e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44870/0 (socket says 172.21.15.116:44870) 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 -- 172.21.15.116:0/2778536449 learned_addr learned my addr 172.21.15.116:0/2778536449 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 -- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5128145250 msgr2=0x7f512813bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51277fe640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5128145250 0x7f512813bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f5126ffd640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51280719d0 0x7f51281583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5128145250 0x7f512813bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 -- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51280719d0 msgr2=0x7f51281583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.389+0000 7f51267fc640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51280719d0 0x7f51281583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f51267fc640 1 -- 172.21.15.116:0/2778536449 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009a70 con 0x7f51280b5690 2024-02-18T11:48:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f51267fc640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 0x7f512813e280 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c500 tx=0x7f511c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c007560 con 0x7f51280b5690 2024-02-18T11:48:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c0047e0 con 0x7f51280b5690 2024-02-18T11:48:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004b90 con 0x7f51280b5690 2024-02-18T11:48:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512813c870 con 0x7f51280b5690 2024-02-18T11:48:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.390+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512813ce30 con 0x7f51280b5690 2024-02-18T11:48:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.391+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51280cf7a0 con 0x7f51280b5690 2024-02-18T11:48:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.391+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f511c004cf0 con 0x7f51280b5690 2024-02-18T11:48:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.391+0000 7f510f7fe640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f511406bca0 0x7f511406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.392+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f511c089cd0 con 0x7f51280b5690 2024-02-18T11:48:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.392+0000 7f5126ffd640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f511406bca0 0x7f511406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.392+0000 7f5126ffd640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f511406bca0 0x7f511406e160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5118036000 tx=0x7f5118030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.396+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f511c0540a0 con 0x7f51280b5690 2024-02-18T11:48:22.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.569+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f512813fad0 con 0x7f51280b5690 2024-02-18T11:48:22.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.569+0000 7f510f7fe640 1 -- 172.21.15.116:0/2778536449 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f511c018020 con 0x7f51280b5690 2024-02-18T11:48:22.563 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f511406bca0 msgr2=0x7f511406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f511406bca0 0x7f511406e160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5118036000 tx=0x7f5118030000 comp rx=0 tx=0).stop 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 msgr2=0x7f512813e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 0x7f512813e280 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c500 tx=0x7f511c00c9c0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 shutdown_connections 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f511406bca0 0x7f511406e160 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5128145250 0x7f512813bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51280b5690 0x7f512813e280 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 --2- 172.21.15.116:0/2778536449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51280719d0 0x7f51281583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 >> 172.21.15.116:0/2778536449 conn(0x7f512806f460 msgr2=0x7f51281417a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:22.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 shutdown_connections 2024-02-18T11:48:22.566 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.571+0000 7f512d348640 1 -- 172.21.15.116:0/2778536449 wait complete. 2024-02-18T11:48:22.576 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-02-18T11:48:22.676 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.683+0000 7fb2b8b2c640 1 Processor -- start 2024-02-18T11:48:22.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.683+0000 7fb2b8b2c640 1 -- start start 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b8b2c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b40cf900 0x7fb2b40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b8b2c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb2b4099a50 0x7fb2b4099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b8b2c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b40d0a80 con 0x7fb2b415efb0 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b8b2c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b409a420 con 0x7fb2b4099a50 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b8b2c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b415f390 con 0x7fb2b40cf900 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb2b4099a50 0x7fb2b4099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb2b4099a50 0x7fb2b4099ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44886/0 (socket says 172.21.15.116:44886) 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 -- 172.21.15.116:0/1380184880 learned_addr learned my addr 172.21.15.116:0/1380184880 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 -- 172.21.15.116:0/1380184880 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b40cf900 msgr2=0x7fb2b40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:22.678 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 --2- 172.21.15.116:0/1380184880 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b40cf900 0x7fb2b40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 -- 172.21.15.116:0/1380184880 >> v1:172.21.15.143:6789/0 conn(0x7fb2b415efb0 legacy=0x7fb2b406c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:22.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.684+0000 7fb2b2d9d640 1 -- 172.21.15.116:0/1380184880 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40cfe70 con 0x7fb2b4099a50 2024-02-18T11:48:22.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.685+0000 7fb2b2d9d640 1 --2- 172.21.15.116:0/1380184880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b4099a50 0x7fb2b4099ee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009e50 tx=0x7fb2ac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d926001b3718909 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.685+0000 7fb2b1d9b640 1 -- 172.21.15.116:0/1380184880 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac02c9e0 con 0x7fb2b4099a50 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.685+0000 7fb2b1d9b640 1 -- 172.21.15.116:0/1380184880 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac02cb40 con 0x7fb2b4099a50 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.685+0000 7fb2b1d9b640 1 -- 172.21.15.116:0/1380184880 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0343f0 con 0x7fb2b4099a50 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/1380184880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b4099a50 msgr2=0x7fb2b4099ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/1380184880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b4099a50 0x7fb2b4099ee0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009e50 tx=0x7fb2ac02be40 comp rx=0 tx=0).stop 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/1380184880 shutdown_connections 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/1380184880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b4099a50 0x7fb2b4099ee0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/1380184880 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b40cf900 0x7fb2b40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/1380184880 >> 172.21.15.116:0/1380184880 conn(0x7fb2b40d1790 msgr2=0x7fb2b40d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/1380184880 shutdown_connections 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.686+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/1380184880 wait complete. 2024-02-18T11:48:22.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.687+0000 7fb2b8b2c640 1 Processor -- start 2024-02-18T11:48:22.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.687+0000 7fb2b8b2c640 1 -- start start 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b8b2c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2b4099a50 0x7fb2b40b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b8b2c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 0x7fb2b40c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b8b2c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b415efb0 0x7fb2b414b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b8b2c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b415d8d0 con 0x7fb2b40cf900 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b8b2c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4156d70 con 0x7fb2b4099a50 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b8b2c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b415ccc0 con 0x7fb2b415efb0 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 0x7fb2b40c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 0x7fb2b40c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44902/0 (socket says 172.21.15.116:44902) 2024-02-18T11:48:22.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 -- 172.21.15.116:0/923208863 learned_addr learned my addr 172.21.15.116:0/923208863 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b259c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2b4099a50 0x7fb2b40b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b2d9d640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b415efb0 0x7fb2b414b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 -- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b415efb0 msgr2=0x7fb2b414b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b415efb0 0x7fb2b414b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 -- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2b4099a50 msgr2=0x7fb2b40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2b4099a50 0x7fb2b40b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 -- 172.21.15.116:0/923208863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac009b30 con 0x7fb2b40cf900 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2b259c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2b4099a50 0x7fb2b40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.688+0000 7fb2abfff640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 0x7fb2b40c9e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b4a0 tx=0x7fb29c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.689+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010610 con 0x7fb2b40cf900 2024-02-18T11:48:22.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.689+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c010770 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.689+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b415cfa0 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.689+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010b20 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.689+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b415d560 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.690+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b4067840 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.690+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fb29c010c80 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.690+0000 7fb2aaffd640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb29406bca0 0x7fb29406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.691+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fb29c08a7e0 con 0x7fb2b40cf900 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.691+0000 7fb2b259c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb29406bca0 0x7fb29406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:22.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.691+0000 7fb2b259c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb29406bca0 0x7fb29406e160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400a830 tx=0x7fb2a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:22.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.695+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb29c054b90 con 0x7fb2b40cf900 2024-02-18T11:48:22.928 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:22.935+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7fb2b4074520 con 0x7fb2b40cf900 2024-02-18T11:48:23.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.726+0000 7fb2aaffd640 1 -- 172.21.15.116:0/923208863 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v68) v1 ==== 147+0+0 (secure 0 0 0) 0x7fb29c058250 con 0x7fb2b40cf900 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb29406bca0 msgr2=0x7fb29406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb29406bca0 0x7fb29406e160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400a830 tx=0x7fb2a4008040 comp rx=0 tx=0).stop 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 msgr2=0x7fb2b40c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 0x7fb2b40c9e00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b4a0 tx=0x7fb29c00b960 comp rx=0 tx=0).stop 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 shutdown_connections 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fb29406bca0 0x7fb29406e160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2b415efb0 0x7fb2b414b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2b40cf900 0x7fb2b40c9e00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 --2- 172.21.15.116:0/923208863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2b4099a50 0x7fb2b40b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 >> 172.21.15.116:0/923208863 conn(0x7fb2b40d1790 msgr2=0x7fb2b4070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 shutdown_connections 2024-02-18T11:48:23.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.729+0000 7fb2b8b2c640 1 -- 172.21.15.116:0/923208863 wait complete. 2024-02-18T11:48:23.736 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-02-18T11:48:23.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.852+0000 7fef23062640 1 Processor -- start 2024-02-18T11:48:23.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.852+0000 7fef23062640 1 -- start start 2024-02-18T11:48:23.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef23062640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef1c150ed0 0x7fef1c0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef23062640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 0x7fef1c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef23062640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c15a480 con 0x7fef1c0b4740 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef23062640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1c15a760 con 0x7fef1c150ed0 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef23062640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c15aa40 con 0x7fef1c0cdd20 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef215ff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fef1c0b4740 0x7fef1c06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53698/0 (socket says 172.21.15.116:53698) 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef215ff640 1 -- 172.21.15.116:0/3958618289 learned_addr learned my addr 172.21.15.116:0/3958618289 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643978786 0 0) 0x7fef1c15a480 con 0x7fef1c0b4740 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef20dfe640 1 --2- 172.21.15.116:0/3958618289 >> v2:172.21.15.143:3300/0 conn(0x7fef1c150ed0 0x7fef1c0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef04003520 con 0x7fef1c0b4740 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.853+0000 7fef1bfff640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 0x7fef1c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 320646639 0 0) 0x7fef04003520 con 0x7fef1c0b4740 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 msgr2=0x7fef1c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:23.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 0x7fef1c071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 >> v2:172.21.15.143:3300/0 conn(0x7fef1c150ed0 msgr2=0x7fef1c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 --2- 172.21.15.116:0/3958618289 >> v2:172.21.15.143:3300/0 conn(0x7fef1c150ed0 0x7fef1c0b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c15ad20 con 0x7fef1c0b4740 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fef140019f0 con 0x7fef1c0b4740 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 >> v1:172.21.15.116:6789/0 conn(0x7fef1c0b4740 legacy=0x7fef1c06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 0x7fef04003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef04003ef0 0x7fef040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c15a760 con 0x7fef04003ef0 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c15aa40 con 0x7fef04004d00 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0400cb90 con 0x7fef1c0cdd20 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1bfff640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef04003ef0 0x7fef040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef215ff640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef20dfe640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 0x7fef04003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1bfff640 1 -- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 msgr2=0x7fef04003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1bfff640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c0cdd20 0x7fef04003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1bfff640 1 -- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04004d00 msgr2=0x7fef04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1bfff640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04004d00 0x7fef04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.854+0000 7fef1bfff640 1 -- 172.21.15.116:0/3958618289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef04003520 con 0x7fef04003ef0 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.855+0000 7fef1bfff640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef04003ef0 0x7fef040047c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef1000ba60 tx=0x7fef040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f115f929f11bf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.855+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10032630 con 0x7fef04003ef0 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.855+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10032790 con 0x7fef04003ef0 2024-02-18T11:48:23.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.855+0000 7fef1b7fe640 1 -- 172.21.15.116:0/3958618289 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10032b20 con 0x7fef04003ef0 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef23062640 1 -- 172.21.15.116:0/3958618289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef04003ef0 msgr2=0x7fef040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef23062640 1 --2- 172.21.15.116:0/3958618289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef04003ef0 0x7fef040047c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fef1000ba60 tx=0x7fef040057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef215ff640 1 -- 172.21.15.116:0/3958618289 reap_dead start 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef23062640 1 -- 172.21.15.116:0/3958618289 shutdown_connections 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef23062640 1 -- 172.21.15.116:0/3958618289 >> 172.21.15.116:0/3958618289 conn(0x7fef1c067000 msgr2=0x7fef1c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef23062640 1 -- 172.21.15.116:0/3958618289 shutdown_connections 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.856+0000 7fef23062640 1 -- 172.21.15.116:0/3958618289 wait complete. 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.857+0000 7fef23062640 1 Processor -- start 2024-02-18T11:48:23.856 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.857+0000 7fef23062640 1 -- start start 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef23062640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04003ef0 0x7fef1c156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef23062640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c159830 0x7fef1c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef23062640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 0x7fef1c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef23062640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c156700 con 0x7fef1c0b1810 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef23062640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0b1c90 con 0x7fef04003ef0 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef23062640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c158d40 con 0x7fef1c159830 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 0x7fef1c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 0x7fef1c158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44920/0 (socket says 172.21.15.116:44920) 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 -- 172.21.15.116:0/3044614066 learned_addr learned my addr 172.21.15.116:0/3044614066 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef20dfe640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04003ef0 0x7fef1c156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef1bfff640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c159830 0x7fef1c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 -- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c159830 msgr2=0x7fef1c0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:23.857 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c159830 0x7fef1c0b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 -- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04003ef0 msgr2=0x7fef1c156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04003ef0 0x7fef1c156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 -- 172.21.15.116:0/3044614066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c155d60 con 0x7fef1c0b1810 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.858+0000 7fef215ff640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 0x7fef1c158010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef14001330 tx=0x7fef14007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.859+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400abc0 con 0x7fef1c0b1810 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.859+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1400ad40 con 0x7fef1c0b1810 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.859+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c156040 con 0x7fef1c0b1810 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.859+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14013630 con 0x7fef1c0b1810 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.859+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c159cb0 con 0x7fef1c0b1810 2024-02-18T11:48:23.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.860+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c074520 con 0x7fef1c0b1810 2024-02-18T11:48:23.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.860+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fef14013790 con 0x7fef1c0b1810 2024-02-18T11:48:23.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.860+0000 7fef197fa640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef0806bc10 0x7fef0806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:23.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.861+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fef1408b0e0 con 0x7fef1c0b1810 2024-02-18T11:48:23.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.861+0000 7fef20dfe640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef0806bc10 0x7fef0806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:23.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.861+0000 7fef20dfe640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef0806bc10 0x7fef0806e0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009b40 tx=0x7fef0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:23.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:23.866+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef14055550 con 0x7fef1c0b1810 2024-02-18T11:48:24.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.035+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fef1c06f150 con 0x7fef1c0b1810 2024-02-18T11:48:24.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.038+0000 7fef197fa640 1 -- 172.21.15.116:0/3044614066 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fef14058c10 con 0x7fef1c0b1810 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.040+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef0806bc10 msgr2=0x7fef0806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.040+0000 7fef23062640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef0806bc10 0x7fef0806e0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009b40 tx=0x7fef0c008040 comp rx=0 tx=0).stop 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.040+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 msgr2=0x7fef1c158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.040+0000 7fef23062640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 0x7fef1c158010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef14001330 tx=0x7fef14007b00 comp rx=0 tx=0).stop 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 shutdown_connections 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fef0806bc10 0x7fef0806e0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c0b1810 0x7fef1c158010 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef1c159830 0x7fef1c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 --2- 172.21.15.116:0/3044614066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef04003ef0 0x7fef1c156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 >> 172.21.15.116:0/3044614066 conn(0x7fef1c067000 msgr2=0x7fef1c0afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 shutdown_connections 2024-02-18T11:48:24.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.041+0000 7fef23062640 1 -- 172.21.15.116:0/3044614066 wait complete. 2024-02-18T11:48:24.045 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-02-18T11:48:24.045 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-18T11:48:24.089 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-02-18T11:48:24.134 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-02-18T11:48:24.134 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:24.134 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:24.149 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:24.150 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:24.205 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:24.205 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:24.260 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:24.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.410+0000 7f2757ec0640 1 Processor -- start 2024-02-18T11:48:24.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.411+0000 7f2757ec0640 1 -- start start 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.411+0000 7f2757ec0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f27500b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.411+0000 7f2757ec0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2750161850 0x7f27500ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.411+0000 7f2757ec0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27500ce780 con 0x7f27500d0970 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.411+0000 7f2757ec0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27500d3730 con 0x7f2750161850 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.411+0000 7f2757ec0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27500cf250 con 0x7f275015dc60 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2750161850 0x7f27500ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2750161850 0x7f27500ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44936/0 (socket says 172.21.15.116:44936) 2024-02-18T11:48:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 -- 172.21.15.116:0/2766556693 learned_addr learned my addr 172.21.15.116:0/2766556693 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f2755c5c640 1 --2- 172.21.15.116:0/2766556693 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f27500b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 -- 172.21.15.116:0/2766556693 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 msgr2=0x7f27500b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 --2- 172.21.15.116:0/2766556693 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f27500b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 -- 172.21.15.116:0/2766556693 >> v1:172.21.15.143:6789/0 conn(0x7f27500d0970 legacy=0x7f27500d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.412+0000 7f275545b640 1 -- 172.21.15.116:0/2766556693 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500ce9b0 con 0x7f2750161850 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f275545b640 1 --2- 172.21.15.116:0/2766556693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2750161850 0x7f27500ce240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f274002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5454c9ba43be1c83 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2754c5a640 1 -- 172.21.15.116:0/2766556693 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740030840 con 0x7f2750161850 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2754c5a640 1 -- 172.21.15.116:0/2766556693 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27400309a0 con 0x7f2750161850 2024-02-18T11:48:24.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2754c5a640 1 -- 172.21.15.116:0/2766556693 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740030d30 con 0x7f2750161850 2024-02-18T11:48:24.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2757ec0640 1 -- 172.21.15.116:0/2766556693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2750161850 msgr2=0x7f27500ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2757ec0640 1 --2- 172.21.15.116:0/2766556693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2750161850 0x7f27500ce240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2740009870 tx=0x7f274002b980 comp rx=0 tx=0).stop 2024-02-18T11:48:24.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2757ec0640 1 -- 172.21.15.116:0/2766556693 shutdown_connections 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2757ec0640 1 --2- 172.21.15.116:0/2766556693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2750161850 0x7f27500ce240 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2757ec0640 1 --2- 172.21.15.116:0/2766556693 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f27500b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.413+0000 7f2757ec0640 1 -- 172.21.15.116:0/2766556693 >> 172.21.15.116:0/2766556693 conn(0x7f27500b5910 msgr2=0x7f27500b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.414+0000 7f2757ec0640 1 -- 172.21.15.116:0/2766556693 shutdown_connections 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.414+0000 7f2757ec0640 1 -- 172.21.15.116:0/2766556693 wait complete. 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 Processor -- start 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 -- start start 2024-02-18T11:48:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 0x7f27501564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f2750159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2750161850 0x7f27501604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2750156a30 con 0x7f27500d0970 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27501f4280 con 0x7f2750161850 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.415+0000 7f2757ec0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27501f4710 con 0x7f275015dc60 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 0x7f27501564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 0x7f27501564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44942/0 (socket says 172.21.15.116:44942) 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 -- 172.21.15.116:0/3024134332 learned_addr learned my addr 172.21.15.116:0/3024134332 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f275645d640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2750161850 0x7f27501604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f275545b640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f2750159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 -- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 msgr2=0x7f2750159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f2750159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 -- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2750161850 msgr2=0x7f27501604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2750161850 0x7f27501604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 -- 172.21.15.116:0/3024134332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27400094d0 con 0x7f27500d0970 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f275645d640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2750161850 0x7f27501604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2755c5c640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 0x7f27501564f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f274c00d800 tx=0x7f274c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c00b6c0 con 0x7f27500d0970 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c00b820 con 0x7f27500d0970 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c00bbb0 con 0x7f27500d0970 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27501f3950 con 0x7f27500d0970 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.416+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501f3f10 con 0x7f27500d0970 2024-02-18T11:48:24.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.417+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2750067840 con 0x7f27500d0970 2024-02-18T11:48:24.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.418+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f274c00bd10 con 0x7f27500d0970 2024-02-18T11:48:24.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.419+0000 7f27467fc640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f273406bc70 0x7f273406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.419+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f274c08b130 con 0x7f27500d0970 2024-02-18T11:48:24.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.419+0000 7f275545b640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f273406bc70 0x7f273406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.419+0000 7f275545b640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f273406bc70 0x7f273406e130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2740036000 tx=0x7f274003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:24.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.423+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f274c0554f0 con 0x7f27500d0970 2024-02-18T11:48:24.594 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.601+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2750074520 con 0x7f27500d0970 2024-02-18T11:48:24.596 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.603+0000 7f27467fc640 1 -- 172.21.15.116:0/3024134332 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f274c058bb0 con 0x7f27500d0970 2024-02-18T11:48:24.596 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:24.596 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:24.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f273406bc70 msgr2=0x7f273406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f273406bc70 0x7f273406e130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2740036000 tx=0x7f274003b040 comp rx=0 tx=0).stop 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 msgr2=0x7f27501564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 0x7f27501564f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f274c00d800 tx=0x7f274c00dcc0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 shutdown_connections 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f273406bc70 0x7f273406e130 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2750161850 0x7f27501604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f275015dc60 0x7f2750159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 --2- 172.21.15.116:0/3024134332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27500d0970 0x7f27501564f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 >> 172.21.15.116:0/3024134332 conn(0x7f27500b5910 msgr2=0x7f275015b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 shutdown_connections 2024-02-18T11:48:24.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.605+0000 7f2757ec0640 1 -- 172.21.15.116:0/3024134332 wait complete. 2024-02-18T11:48:24.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:24.611 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:24.717 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.724+0000 7efd4ae39640 1 Processor -- start 2024-02-18T11:48:24.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.724+0000 7efd4ae39640 1 -- start start 2024-02-18T11:48:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd4ae39640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efd4415dbf0 0x7efd441617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd4ae39640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efd440cee50 0x7efd440d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd4ae39640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440d1ee0 con 0x7efd440b9370 2024-02-18T11:48:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd4ae39640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd44161dc0 con 0x7efd440cee50 2024-02-18T11:48:24.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd4ae39640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd441620a0 con 0x7efd4415dbf0 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efd440cee50 0x7efd440d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efd440cee50 0x7efd440d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44952/0 (socket says 172.21.15.116:44952) 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 -- 172.21.15.116:0/2035237802 learned_addr learned my addr 172.21.15.116:0/2035237802 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd48bd5640 1 --2- 172.21.15.116:0/2035237802 >> v2:172.21.15.143:3300/0 conn(0x7efd4415dbf0 0x7efd441617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 -- 172.21.15.116:0/2035237802 >> v2:172.21.15.143:3300/0 conn(0x7efd4415dbf0 msgr2=0x7efd441617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 --2- 172.21.15.116:0/2035237802 >> v2:172.21.15.143:3300/0 conn(0x7efd4415dbf0 0x7efd441617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 -- 172.21.15.116:0/2035237802 >> v1:172.21.15.143:6789/0 conn(0x7efd440b9370 legacy=0x7efd440ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.720 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.725+0000 7efd493d6640 1 -- 172.21.15.116:0/2035237802 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44162380 con 0x7efd440cee50 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.727+0000 7efd493d6640 1 --2- 172.21.15.116:0/2035237802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd440cee50 0x7efd440d2370 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd4406d330 tx=0x7efd3c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdab3c637374b93 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.727+0000 7efd437fe640 1 -- 172.21.15.116:0/2035237802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c02cb20 con 0x7efd440cee50 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.727+0000 7efd437fe640 1 -- 172.21.15.116:0/2035237802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c02cc80 con 0x7efd440cee50 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.727+0000 7efd437fe640 1 -- 172.21.15.116:0/2035237802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c0343f0 con 0x7efd440cee50 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 -- 172.21.15.116:0/2035237802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd440cee50 msgr2=0x7efd440d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 --2- 172.21.15.116:0/2035237802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd440cee50 0x7efd440d2370 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd4406d330 tx=0x7efd3c02bf80 comp rx=0 tx=0).stop 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 -- 172.21.15.116:0/2035237802 shutdown_connections 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 --2- 172.21.15.116:0/2035237802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd440cee50 0x7efd440d2370 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 --2- 172.21.15.116:0/2035237802 >> v2:172.21.15.143:3300/0 conn(0x7efd4415dbf0 0x7efd441617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 -- 172.21.15.116:0/2035237802 >> 172.21.15.116:0/2035237802 conn(0x7efd440b5480 msgr2=0x7efd440b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 -- 172.21.15.116:0/2035237802 shutdown_connections 2024-02-18T11:48:24.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.728+0000 7efd4ae39640 1 -- 172.21.15.116:0/2035237802 wait complete. 2024-02-18T11:48:24.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.730+0000 7efd4ae39640 1 Processor -- start 2024-02-18T11:48:24.723 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.730+0000 7efd4ae39640 1 -- start start 2024-02-18T11:48:24.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd4ae39640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efd440b9370 0x7efd441546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd4ae39640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd440cee50 0x7efd440b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd4ae39640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 0x7efd440b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd4ae39640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4415f360 con 0x7efd4415dbf0 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd4ae39640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44154c10 con 0x7efd440cee50 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd4ae39640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4415e750 con 0x7efd440b9370 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd493d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 0x7efd440b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd48bd5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efd440b9370 0x7efd441546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd493d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 0x7efd440b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44964/0 (socket says 172.21.15.116:44964) 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd493d6640 1 -- 172.21.15.116:0/3891375019 learned_addr learned my addr 172.21.15.116:0/3891375019 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.731+0000 7efd43fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd440cee50 0x7efd440b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd493d6640 1 -- 172.21.15.116:0/3891375019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efd440b9370 msgr2=0x7efd441546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd493d6640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efd440b9370 0x7efd441546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd493d6640 1 -- 172.21.15.116:0/3891375019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd440cee50 msgr2=0x7efd440b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd493d6640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd440cee50 0x7efd440b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd493d6640 1 -- 172.21.15.116:0/3891375019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c009b30 con 0x7efd4415dbf0 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd493d6640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 0x7efd440b80a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd3c036040 tx=0x7efd3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c009510 con 0x7efd4415dbf0 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c02c920 con 0x7efd4415dbf0 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4415e150 con 0x7efd4415dbf0 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c03e690 con 0x7efd4415dbf0 2024-02-18T11:48:24.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.732+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4415ed10 con 0x7efd4415dbf0 2024-02-18T11:48:24.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.733+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd44067840 con 0x7efd4415dbf0 2024-02-18T11:48:24.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.734+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7efd3c03e7f0 con 0x7efd4415dbf0 2024-02-18T11:48:24.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.734+0000 7efd417fa640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7efd2806bc70 0x7efd2806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:24.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.734+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7efd3c0ad480 con 0x7efd4415dbf0 2024-02-18T11:48:24.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.734+0000 7efd48bd5640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7efd2806bc70 0x7efd2806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:24.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.734+0000 7efd48bd5640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7efd2806bc70 0x7efd2806e130 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd38009b40 tx=0x7efd38009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:24.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.738+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd3c077980 con 0x7efd4415dbf0 2024-02-18T11:48:24.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.921+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd44074520 con 0x7efd4415dbf0 2024-02-18T11:48:24.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.921+0000 7efd417fa640 1 -- 172.21.15.116:0/3891375019 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd3c07b040 con 0x7efd4415dbf0 2024-02-18T11:48:24.915 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.923+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7efd2806bc70 msgr2=0x7efd2806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.923+0000 7efd4ae39640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7efd2806bc70 0x7efd2806e130 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efd38009b40 tx=0x7efd38009190 comp rx=0 tx=0).stop 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.923+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 msgr2=0x7efd440b80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.923+0000 7efd4ae39640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 0x7efd440b80a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd3c036040 tx=0x7efd3c004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 shutdown_connections 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7efd2806bc70 0x7efd2806e130 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4415dbf0 0x7efd440b80a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd440cee50 0x7efd440b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 --2- 172.21.15.116:0/3891375019 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efd440b9370 0x7efd441546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 >> 172.21.15.116:0/3891375019 conn(0x7efd440b5480 msgr2=0x7efd44158190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 shutdown_connections 2024-02-18T11:48:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:24.924+0000 7efd4ae39640 1 -- 172.21.15.116:0/3891375019 wait complete. 2024-02-18T11:48:24.928 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:25.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.040+0000 7f2ef9b27640 1 Processor -- start 2024-02-18T11:48:25.033 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.040+0000 7f2ef9b27640 1 -- start start 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef9b27640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef40711c0 0x7f2ef4071620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef9b27640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ef415da30 0x7f2ef415de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef9b27640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef4071b60 con 0x7f2ef4145230 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef9b27640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef415d050 con 0x7f2ef415da30 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef9b27640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4145610 con 0x7f2ef40711c0 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef37fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2ef4145230 0x7f2ef415be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53708/0 (socket says 172.21.15.116:53708) 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef37fe640 1 -- 172.21.15.116:0/3128842363 learned_addr learned my addr 172.21.15.116:0/3128842363 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/3128842363 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef40711c0 0x7f2ef4071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef3fff640 1 --2- 172.21.15.116:0/3128842363 >> v2:172.21.15.143:3300/0 conn(0x7f2ef415da30 0x7f2ef415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef27fc640 1 -- 172.21.15.116:0/3128842363 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622506304 0 0) 0x7f2ef4071b60 con 0x7f2ef4145230 2024-02-18T11:48:25.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef27fc640 1 -- 172.21.15.116:0/3128842363 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ee0003520 con 0x7f2ef4145230 2024-02-18T11:48:25.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef2ffd640 1 -- 172.21.15.116:0/3128842363 >> v2:172.21.15.143:3300/0 conn(0x7f2ef415da30 msgr2=0x7f2ef415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:25.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/3128842363 >> v2:172.21.15.143:3300/0 conn(0x7f2ef415da30 0x7f2ef415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef2ffd640 1 -- 172.21.15.116:0/3128842363 >> v1:172.21.15.116:6789/0 conn(0x7f2ef4145230 legacy=0x7f2ef415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.042+0000 7f2ef2ffd640 1 -- 172.21.15.116:0/3128842363 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef415c960 con 0x7f2ef40711c0 2024-02-18T11:48:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/3128842363 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef40711c0 0x7f2ef4071620 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0097a0 tx=0x7f2edc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78b7356ea58ac34 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef27fc640 1 -- 172.21.15.116:0/3128842363 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc031830 con 0x7f2ef40711c0 2024-02-18T11:48:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef27fc640 1 -- 172.21.15.116:0/3128842363 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc031990 con 0x7f2ef40711c0 2024-02-18T11:48:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef27fc640 1 -- 172.21.15.116:0/3128842363 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc031d20 con 0x7f2ef40711c0 2024-02-18T11:48:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 -- 172.21.15.116:0/3128842363 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef40711c0 msgr2=0x7f2ef4071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/3128842363 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef40711c0 0x7f2ef4071620 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0097a0 tx=0x7f2edc02b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 -- 172.21.15.116:0/3128842363 shutdown_connections 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/3128842363 >> v2:172.21.15.143:3300/0 conn(0x7f2ef415da30 0x7f2ef415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/3128842363 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef40711c0 0x7f2ef4071620 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 -- 172.21.15.116:0/3128842363 >> 172.21.15.116:0/3128842363 conn(0x7f2ef4067000 msgr2=0x7f2ef406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 -- 172.21.15.116:0/3128842363 shutdown_connections 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.043+0000 7f2ef9b27640 1 -- 172.21.15.116:0/3128842363 wait complete. 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.045+0000 7f2ef9b27640 1 Processor -- start 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.045+0000 7f2ef9b27640 1 -- start start 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 0x7f2ef413b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ef4145230 0x7f2ef41387f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef415da30 0x7f2ef4139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4139a70 con 0x7f2ef40711c0 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef413aa40 con 0x7f2ef4145230 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef4138e60 con 0x7f2ef415da30 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 0x7f2ef413b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 0x7f2ef413b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44974/0 (socket says 172.21.15.116:44974) 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 -- 172.21.15.116:0/1031706047 learned_addr learned my addr 172.21.15.116:0/1031706047 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ef4145230 0x7f2ef41387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef3fff640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef415da30 0x7f2ef4139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 -- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef415da30 msgr2=0x7f2ef4139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef415da30 0x7f2ef4139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 -- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ef4145230 msgr2=0x7f2ef41387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ef4145230 0x7f2ef41387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 -- 172.21.15.116:0/1031706047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc009400 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ef4145230 0x7f2ef41387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef3fff640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef415da30 0x7f2ef4139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef37fe640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 0x7f2ef413b640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400cb40 tx=0x7f2ee400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400aca0 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee400ae00 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef4139140 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4012620 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.046+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef4139700 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.048+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef4074520 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.048+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f2ee4007070 con 0x7f2ef40711c0 2024-02-18T11:48:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.049+0000 7f2eebfff640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2ed406bc70 0x7f2ed406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.049+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2ed406bc70 0x7f2ed406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.049+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2ee408abe0 con 0x7f2ef40711c0 2024-02-18T11:48:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.049+0000 7f2ef2ffd640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2ed406bc70 0x7f2ed406e130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2edc036000 tx=0x7f2edc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.053+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ee4055030 con 0x7f2ef40711c0 2024-02-18T11:48:25.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.244+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ef4142b00 con 0x7f2ef40711c0 2024-02-18T11:48:25.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.245+0000 7f2eebfff640 1 -- 172.21.15.116:0/1031706047 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ee40170a0 con 0x7f2ef40711c0 2024-02-18T11:48:25.237 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:25.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2ed406bc70 msgr2=0x7f2ed406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2ed406bc70 0x7f2ed406e130 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2edc036000 tx=0x7f2edc03c040 comp rx=0 tx=0).stop 2024-02-18T11:48:25.239 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 msgr2=0x7f2ef413b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 0x7f2ef413b640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400cb40 tx=0x7f2ee400a060 comp rx=0 tx=0).stop 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 shutdown_connections 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2ed406bc70 0x7f2ed406e130 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2ef415da30 0x7f2ef4139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ef4145230 0x7f2ef41387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 --2- 172.21.15.116:0/1031706047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef40711c0 0x7f2ef413b640 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 >> 172.21.15.116:0/1031706047 conn(0x7f2ef4067000 msgr2=0x7f2ef40cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 shutdown_connections 2024-02-18T11:48:25.240 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.247+0000 7f2ef9b27640 1 -- 172.21.15.116:0/1031706047 wait complete. 2024-02-18T11:48:25.250 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:25.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.362+0000 7fa15ecb5640 1 Processor -- start 2024-02-18T11:48:25.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.362+0000 7fa15ecb5640 1 -- start start 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ecb5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa158072db0 0x7fa15806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ecb5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa15815f3a0 0x7fa1580cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ecb5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa158070cd0 con 0x7fa15815b880 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ecb5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1580cffa0 con 0x7fa15815f3a0 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ecb5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15815bc60 con 0x7fa158072db0 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ca51640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa15815b880 0x7fa1580726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53722/0 (socket says 172.21.15.116:53722) 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15ca51640 1 -- 172.21.15.116:0/152056039 learned_addr learned my addr 172.21.15.116:0/152056039 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa15d252640 1 --2- 172.21.15.116:0/152056039 >> v2:172.21.15.143:3300/0 conn(0x7fa15815f3a0 0x7fa1580cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa157fff640 1 --2- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa158072db0 0x7fa15806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa1577fe640 1 -- 172.21.15.116:0/152056039 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896785074 0 0) 0x7fa158070cd0 con 0x7fa15815b880 2024-02-18T11:48:25.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.363+0000 7fa1577fe640 1 -- 172.21.15.116:0/152056039 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa144003520 con 0x7fa15815b880 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.364+0000 7fa15d252640 1 -- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa158072db0 msgr2=0x7fa15806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.364+0000 7fa15d252640 1 --2- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa158072db0 0x7fa15806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.364+0000 7fa15d252640 1 -- 172.21.15.116:0/152056039 >> v1:172.21.15.116:6789/0 conn(0x7fa15815b880 legacy=0x7fa1580726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.364+0000 7fa1577fe640 1 -- 172.21.15.116:0/152056039 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 662570836 0 0) 0x7fa144003520 con 0x7fa15815b880 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.364+0000 7fa15d252640 1 -- 172.21.15.116:0/152056039 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580cf6b0 con 0x7fa15815f3a0 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.364+0000 7fa15d252640 1 --2- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815f3a0 0x7fa1580cd470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa150009d90 tx=0x7fa15002bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36cf62cbf781ae7d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa1577fe640 1 -- 172.21.15.116:0/152056039 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148001670 con 0x7fa15815f3a0 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa1577fe640 1 -- 172.21.15.116:0/152056039 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15002c8a0 con 0x7fa15815f3a0 2024-02-18T11:48:25.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa1577fe640 1 -- 172.21.15.116:0/152056039 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15002cc10 con 0x7fa15815f3a0 2024-02-18T11:48:25.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 -- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815f3a0 msgr2=0x7fa1580cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815f3a0 0x7fa1580cd470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa150009d90 tx=0x7fa15002bd70 comp rx=0 tx=0).stop 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 -- 172.21.15.116:0/152056039 shutdown_connections 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815f3a0 0x7fa1580cd470 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/152056039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa158072db0 0x7fa15806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 -- 172.21.15.116:0/152056039 >> 172.21.15.116:0/152056039 conn(0x7fa1580b23f0 msgr2=0x7fa1580b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 -- 172.21.15.116:0/152056039 shutdown_connections 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.365+0000 7fa15ecb5640 1 -- 172.21.15.116:0/152056039 wait complete. 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.366+0000 7fa15ecb5640 1 Processor -- start 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.366+0000 7fa15ecb5640 1 -- start start 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ecb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 0x7fa15815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ecb5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815b880 0x7fa1580cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ecb5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa15815f3a0 0x7fa1580ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ecb5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ad340 con 0x7fa158072db0 2024-02-18T11:48:25.362 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ecb5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1580ad620 con 0x7fa15815b880 2024-02-18T11:48:25.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ecb5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa158159520 con 0x7fa15815f3a0 2024-02-18T11:48:25.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ca51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 0x7fa15815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ca51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 0x7fa15815a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44988/0 (socket says 172.21.15.116:44988) 2024-02-18T11:48:25.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.367+0000 7fa15ca51640 1 -- 172.21.15.116:0/3927929910 learned_addr learned my addr 172.21.15.116:0/3927929910 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15d252640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa15815f3a0 0x7fa1580ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa157fff640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815b880 0x7fa1580cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ca51640 1 -- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa15815f3a0 msgr2=0x7fa1580ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ca51640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa15815f3a0 0x7fa1580ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ca51640 1 -- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815b880 msgr2=0x7fa1580cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ca51640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815b880 0x7fa1580cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ca51640 1 -- 172.21.15.116:0/3927929910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150009a70 con 0x7fa158072db0 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15d252640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa15815f3a0 0x7fa1580ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa157fff640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815b880 0x7fa1580cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ca51640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 0x7fa15815a000 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c750 tx=0x7fa14800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148005ad0 con 0x7fa158072db0 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa148005c50 con 0x7fa158072db0 2024-02-18T11:48:25.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa158159800 con 0x7fa158072db0 2024-02-18T11:48:25.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa148007070 con 0x7fa158072db0 2024-02-18T11:48:25.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.368+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15815aea0 con 0x7fa158072db0 2024-02-18T11:48:25.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.369+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa158159380 con 0x7fa158072db0 2024-02-18T11:48:25.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.370+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa1480071d0 con 0x7fa158072db0 2024-02-18T11:48:25.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.370+0000 7fa1557fa640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa13806bc70 0x7fa13806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.370+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fa14808a8f0 con 0x7fa158072db0 2024-02-18T11:48:25.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.370+0000 7fa157fff640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa13806bc70 0x7fa13806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.370+0000 7fa157fff640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa13806bc70 0x7fa13806e130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa1580cd170 tx=0x7fa140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.368 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.374+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa158159380 con 0x7fa158072db0 2024-02-18T11:48:25.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.552+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa158159380 con 0x7fa158072db0 2024-02-18T11:48:25.544 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.552+0000 7fa1557fa640 1 -- 172.21.15.116:0/3927929910 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa158159380 con 0x7fa158072db0 2024-02-18T11:48:25.545 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa13806bc70 msgr2=0x7fa13806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa13806bc70 0x7fa13806e130 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa1580cd170 tx=0x7fa140008040 comp rx=0 tx=0).stop 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 msgr2=0x7fa15815a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 0x7fa15815a000 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa14800c750 tx=0x7fa14800cc10 comp rx=0 tx=0).stop 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 shutdown_connections 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa13806bc70 0x7fa13806e130 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa15815f3a0 0x7fa1580ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15815b880 0x7fa1580cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 --2- 172.21.15.116:0/3927929910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa158072db0 0x7fa15815a000 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 >> 172.21.15.116:0/3927929910 conn(0x7fa1580b23f0 msgr2=0x7fa15815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 shutdown_connections 2024-02-18T11:48:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.554+0000 7fa15ecb5640 1 -- 172.21.15.116:0/3927929910 wait complete. 2024-02-18T11:48:25.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:25.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.667+0000 7f74037e2640 1 Processor -- start 2024-02-18T11:48:25.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.668+0000 7f74037e2640 1 -- start start 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.668+0000 7f74037e2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73fc145230 0x7f73fc15be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.668+0000 7f74037e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 0x7f73fc0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.668+0000 7f74037e2640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc071b50 con 0x7f73fc071700 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.668+0000 7f74037e2640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73fc15d050 con 0x7f73fc145230 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.668+0000 7f74037e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc145610 con 0x7f73fc15c3d0 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 0x7f73fc0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 0x7f73fc0711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:55038/0 (socket says 172.21.15.116:55038) 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 -- 172.21.15.116:0/1474641377 learned_addr learned my addr 172.21.15.116:0/1474641377 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f740157e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73fc145230 0x7f73fc15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 -- 172.21.15.116:0/1474641377 >> v2:172.21.15.143:3300/0 conn(0x7f73fc145230 msgr2=0x7f73fc15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f73f3fff640 1 -- 172.21.15.116:0/1474641377 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606399446 0 0) 0x7f73f4001490 con 0x7f73fc071700 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 --2- 172.21.15.116:0/1474641377 >> v2:172.21.15.143:3300/0 conn(0x7f73fc145230 0x7f73fc15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 -- 172.21.15.116:0/1474641377 >> v1:172.21.15.143:6789/0 conn(0x7f73fc071700 legacy=0x7f73fc06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.669+0000 7f7400d7d640 1 -- 172.21.15.116:0/1474641377 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc15c960 con 0x7f73fc15c3d0 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.670+0000 7f7400d7d640 1 --2- 172.21.15.116:0/1474641377 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 0x7f73fc0711c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009870 tx=0x7f73f802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a7b873b3deaa1a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.670+0000 7f73f3fff640 1 -- 172.21.15.116:0/1474641377 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8031640 con 0x7f73fc15c3d0 2024-02-18T11:48:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.670+0000 7f73f3fff640 1 -- 172.21.15.116:0/1474641377 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f80317a0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.670+0000 7f73f3fff640 1 -- 172.21.15.116:0/1474641377 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f8031b30 con 0x7f73fc15c3d0 2024-02-18T11:48:25.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 -- 172.21.15.116:0/1474641377 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 msgr2=0x7f73fc0711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 --2- 172.21.15.116:0/1474641377 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 0x7f73fc0711c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009870 tx=0x7f73f802b8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 -- 172.21.15.116:0/1474641377 shutdown_connections 2024-02-18T11:48:25.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 --2- 172.21.15.116:0/1474641377 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc15c3d0 0x7f73fc0711c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 --2- 172.21.15.116:0/1474641377 >> v2:172.21.15.143:3300/0 conn(0x7f73fc145230 0x7f73fc15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 -- 172.21.15.116:0/1474641377 >> 172.21.15.116:0/1474641377 conn(0x7f73fc067000 msgr2=0x7f73fc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 -- 172.21.15.116:0/1474641377 shutdown_connections 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.671+0000 7f74037e2640 1 -- 172.21.15.116:0/1474641377 wait complete. 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.672+0000 7f74037e2640 1 Processor -- start 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.672+0000 7f74037e2640 1 -- start start 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc071700 0x7f73fc0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc145230 0x7f73fc15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 0x7f73fc15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0ac680 con 0x7f73fc15c3d0 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0ac960 con 0x7f73fc071700 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc0acc40 con 0x7f73fc145230 2024-02-18T11:48:25.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 0x7f73fc15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 0x7f73fc15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44996/0 (socket says 172.21.15.116:44996) 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 -- 172.21.15.116:0/2746635497 learned_addr learned my addr 172.21.15.116:0/2746635497 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f740157e640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc071700 0x7f73fc0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7400d7d640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc145230 0x7f73fc15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 -- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc145230 msgr2=0x7f73fc15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc145230 0x7f73fc15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 -- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc071700 msgr2=0x7f73fc0ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc071700 0x7f73fc0ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 -- 172.21.15.116:0/2746635497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f80094d0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7400d7d640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc145230 0x7f73fc15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f7401d7f640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 0x7f73fc15d930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c780 tx=0x7f73f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4003da0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400ce90 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc0acf20 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.673+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4005290 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.674+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc13f830 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.674+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc06baf0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.675+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f73f40053f0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.675+0000 7f73f1ffb640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e006bc70 0x7f73e006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.675+0000 7f740157e640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e006bc70 0x7f73e006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.675+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f73f408a1b0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.671 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.676+0000 7f740157e640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e006bc70 0x7f73e006e130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009b40 tx=0x7f73ec009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.680+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73f40546b0 con 0x7f73fc15c3d0 2024-02-18T11:48:25.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.849+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f73fc071b50 con 0x7f73fc15c3d0 2024-02-18T11:48:25.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.850+0000 7f73f1ffb640 1 -- 172.21.15.116:0/2746635497 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f73fc071b50 con 0x7f73fc15c3d0 2024-02-18T11:48:25.842 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:25.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e006bc70 msgr2=0x7f73e006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e006bc70 0x7f73e006e130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73ec009b40 tx=0x7f73ec009190 comp rx=0 tx=0).stop 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 msgr2=0x7f73fc15d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 0x7f73fc15d930 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f73f400c780 tx=0x7f73f400cc40 comp rx=0 tx=0).stop 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 shutdown_connections 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f73e006bc70 0x7f73e006e130 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73fc15c3d0 0x7f73fc15d930 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73fc145230 0x7f73fc15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 --2- 172.21.15.116:0/2746635497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73fc071700 0x7f73fc0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 >> 172.21.15.116:0/2746635497 conn(0x7f73fc067000 msgr2=0x7f73fc141f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 shutdown_connections 2024-02-18T11:48:25.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.852+0000 7f74037e2640 1 -- 172.21.15.116:0/2746635497 wait complete. 2024-02-18T11:48:25.855 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:48:25.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.963+0000 7fe38a2fb640 1 Processor -- start 2024-02-18T11:48:25.956 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.963+0000 7fe38a2fb640 1 -- start start 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe38a2fb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe38406ebf0 0x7fe38406f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe38a2fb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840cf980 con 0x7fe38415dc90 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe38a2fb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38406ab60 con 0x7fe3840b2db0 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe38a2fb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38415e070 con 0x7fe38406ebf0 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe3837fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe38406ebf0 0x7fe38406f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe383fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe38415dc90 0x7fe3840ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53738/0 (socket says 172.21.15.116:53738) 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.964+0000 7fe383fff640 1 -- 172.21.15.116:0/1106622085 learned_addr learned my addr 172.21.15.116:0/1106622085 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.965+0000 7fe3837fe640 1 -- 172.21.15.116:0/1106622085 >> v1:172.21.15.143:6789/0 conn(0x7fe3840b2db0 legacy=0x7fe3840b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.965+0000 7fe3837fe640 1 -- 172.21.15.116:0/1106622085 >> v1:172.21.15.116:6789/0 conn(0x7fe38415dc90 legacy=0x7fe3840ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.965+0000 7fe3837fe640 1 -- 172.21.15.116:0/1106622085 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38406ae40 con 0x7fe38406ebf0 2024-02-18T11:48:25.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.965+0000 7fe382ffd640 1 -- 172.21.15.116:0/1106622085 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359864447 0 0) 0x7fe3840cf980 con 0x7fe38415dc90 2024-02-18T11:48:25.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.965+0000 7fe3837fe640 1 --2- 172.21.15.116:0/1106622085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38406ebf0 0x7fe38406f080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe3780097a0 tx=0x7fe37802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a7db49f1bfae10 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe382ffd640 1 -- 172.21.15.116:0/1106622085 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378031830 con 0x7fe38406ebf0 2024-02-18T11:48:25.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe382ffd640 1 -- 172.21.15.116:0/1106622085 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe378031990 con 0x7fe38406ebf0 2024-02-18T11:48:25.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe382ffd640 1 -- 172.21.15.116:0/1106622085 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378031d20 con 0x7fe38406ebf0 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 -- 172.21.15.116:0/1106622085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38406ebf0 msgr2=0x7fe38406f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/1106622085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38406ebf0 0x7fe38406f080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe3780097a0 tx=0x7fe37802b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 -- 172.21.15.116:0/1106622085 shutdown_connections 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/1106622085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38406ebf0 0x7fe38406f080 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 -- 172.21.15.116:0/1106622085 >> 172.21.15.116:0/1106622085 conn(0x7fe3840cfd80 msgr2=0x7fe3840d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 -- 172.21.15.116:0/1106622085 shutdown_connections 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.966+0000 7fe38a2fb640 1 -- 172.21.15.116:0/1106622085 wait complete. 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.967+0000 7fe38a2fb640 1 Processor -- start 2024-02-18T11:48:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.967+0000 7fe38a2fb640 1 -- start start 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe38a2fb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe38406ebf0 0x7fe38415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe38a2fb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3840b2db0 0x7fe3840ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe38a2fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 0x7fe3841f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe38a2fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38415f130 con 0x7fe38415dc90 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe38a2fb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840aeca0 con 0x7fe3840b2db0 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe38a2fb640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38415f320 con 0x7fe38406ebf0 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe388898640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 0x7fe3841f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe388898640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 0x7fe3841f2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45016/0 (socket says 172.21.15.116:45016) 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe388898640 1 -- 172.21.15.116:0/3807286435 learned_addr learned my addr 172.21.15.116:0/3807286435 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe3837fe640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3840b2db0 0x7fe3840ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.968+0000 7fe383fff640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe38406ebf0 0x7fe38415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe388898640 1 -- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe38406ebf0 msgr2=0x7fe38415ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe388898640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe38406ebf0 0x7fe38415ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe388898640 1 -- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3840b2db0 msgr2=0x7fe3840ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe388898640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3840b2db0 0x7fe3840ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe388898640 1 -- 172.21.15.116:0/3807286435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe378009400 con 0x7fe38415dc90 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe3837fe640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3840b2db0 0x7fe3840ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe383fff640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe38406ebf0 0x7fe38415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe388898640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 0x7fe3841f2ad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00c780 tx=0x7fe37c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c004b10 con 0x7fe38415dc90 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c004c70 con 0x7fe38415dc90 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38415f660 con 0x7fe38415dc90 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00aa90 con 0x7fe38415dc90 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.969+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38415fea0 con 0x7fe38415dc90 2024-02-18T11:48:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.970+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe384067180 con 0x7fe38415dc90 2024-02-18T11:48:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.970+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe37c00abf0 con 0x7fe38415dc90 2024-02-18T11:48:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.971+0000 7fe380ff9640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe37006bc70 0x7fe37006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.971+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe37c08af70 con 0x7fe38415dc90 2024-02-18T11:48:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.971+0000 7fe383fff640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe37006bc70 0x7fe37006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.971+0000 7fe383fff640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe37006bc70 0x7fe37006e130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe374000f60 tx=0x7fe374007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:25.967 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:25.975+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe37c0be8c0 con 0x7fe38415dc90 2024-02-18T11:48:26.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.157+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe384160090 con 0x7fe38415dc90 2024-02-18T11:48:26.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.157+0000 7fe380ff9640 1 -- 172.21.15.116:0/3807286435 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe37c055330 con 0x7fe38415dc90 2024-02-18T11:48:26.150 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T11:48:26.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.159+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe37006bc70 msgr2=0x7fe37006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.159+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe37006bc70 0x7fe37006e130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe374000f60 tx=0x7fe374007c10 comp rx=0 tx=0).stop 2024-02-18T11:48:26.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.159+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 msgr2=0x7fe3841f2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.159+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 0x7fe3841f2ad0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00c780 tx=0x7fe37c00cc40 comp rx=0 tx=0).stop 2024-02-18T11:48:26.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 shutdown_connections 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe37006bc70 0x7fe37006e130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38415dc90 0x7fe3841f2ad0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3840b2db0 0x7fe3840ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 --2- 172.21.15.116:0/3807286435 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe38406ebf0 0x7fe38415ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 >> 172.21.15.116:0/3807286435 conn(0x7fe3840cfd80 msgr2=0x7fe3840afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 shutdown_connections 2024-02-18T11:48:26.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.160+0000 7fe38a2fb640 1 -- 172.21.15.116:0/3807286435 wait complete. 2024-02-18T11:48:26.163 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-18T11:48:26.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:26.163 DEBUG:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-02-18T11:48:26.192 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:48:26.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:26.192 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-18T11:48:26.247 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:26.247 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:26.267 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:26.267 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:26.324 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:26.324 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:26.381 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:26.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.490+0000 7f3fc32c3640 1 Processor -- start 2024-02-18T11:48:26.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc32c3640 1 -- start start 2024-02-18T11:48:26.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc32c3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3fbc15dc60 0x7f3fbc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc32c3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc32c3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc0ce780 con 0x7f3fbc161850 2024-02-18T11:48:26.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc32c3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc0d3730 con 0x7f3fbc15dc60 2024-02-18T11:48:26.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc32c3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc0cf250 con 0x7f3fbc0d1080 2024-02-18T11:48:26.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc105f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3fbc15dc60 0x7f3fbc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc105f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3fbc15dc60 0x7f3fbc0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45032/0 (socket says 172.21.15.116:45032) 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc105f640 1 -- 172.21.15.116:0/1827634225 learned_addr learned my addr 172.21.15.116:0/1827634225 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.491+0000 7f3fc085e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3fbc161850 0x7f3fbc0d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53750/0 (socket says 172.21.15.116:53750) 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fc1860640 1 --2- 172.21.15.116:0/1827634225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fc105f640 1 -- 172.21.15.116:0/1827634225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 msgr2=0x7f3fbc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fc105f640 1 --2- 172.21.15.116:0/1827634225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fc105f640 1 -- 172.21.15.116:0/1827634225 >> v1:172.21.15.116:6789/0 conn(0x7f3fbc161850 legacy=0x7f3fbc0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fc105f640 1 -- 172.21.15.116:0/1827634225 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0ce9b0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fb3fff640 1 -- 172.21.15.116:0/1827634225 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605869110 0 0) 0x7f3fbc0ce780 con 0x7f3fbc161850 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.492+0000 7f3fc105f640 1 --2- 172.21.15.116:0/1827634225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc0b6c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009e50 tx=0x7f3fb802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29fd3ad2b881f8b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fb3fff640 1 -- 172.21.15.116:0/1827634225 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb802cb00 con 0x7f3fbc15dc60 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fb3fff640 1 -- 172.21.15.116:0/1827634225 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb802cc60 con 0x7f3fbc15dc60 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fb3fff640 1 -- 172.21.15.116:0/1827634225 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80343f0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 -- 172.21.15.116:0/1827634225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 msgr2=0x7f3fbc0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/1827634225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc0b6c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009e50 tx=0x7f3fb802bf60 comp rx=0 tx=0).stop 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 -- 172.21.15.116:0/1827634225 shutdown_connections 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/1827634225 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/1827634225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc0b6c10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 -- 172.21.15.116:0/1827634225 >> 172.21.15.116:0/1827634225 conn(0x7f3fbc0b5910 msgr2=0x7f3fbc0b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 -- 172.21.15.116:0/1827634225 shutdown_connections 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.493+0000 7f3fc32c3640 1 -- 172.21.15.116:0/1827634225 wait complete. 2024-02-18T11:48:26.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.494+0000 7f3fc32c3640 1 Processor -- start 2024-02-18T11:48:26.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.494+0000 7f3fc32c3640 1 -- start start 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc32c3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc32c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc32c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc161850 0x7f3fbc160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc32c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc15f7e0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc32c3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc160640 con 0x7f3fbc161850 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc32c3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc151720 con 0x7f3fbc0d1080 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc15ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45036/0 (socket says 172.21.15.116:45036) 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 -- 172.21.15.116:0/4268438208 learned_addr learned my addr 172.21.15.116:0/4268438208 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 -- 172.21.15.116:0/4268438208 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 msgr2=0x7f3fbc1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 -- 172.21.15.116:0/4268438208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc161850 msgr2=0x7f3fbc160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc161850 0x7f3fbc160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.495+0000 7f3fc085e640 1 -- 172.21.15.116:0/4268438208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb8009b30 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.496+0000 7f3fc085e640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc15ed10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c420 tx=0x7f3fac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.496+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac0074e0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.496+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac007640 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.496+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc15fa30 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.496+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac004a00 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.496+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc15fff0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.497+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc067840 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.497+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3fac004b60 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.498+0000 7f3fb1ffb640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3fa006bc70 0x7f3fa006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.498+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3fac08a9a0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.491 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.498+0000 7f3fc105f640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3fa006bc70 0x7f3fa006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.498+0000 7f3fc105f640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3fa006bc70 0x7f3fa006e130 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8035000 tx=0x7f3fb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:26.494 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.502+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fac054df0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.669 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.677+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fbc074520 con 0x7f3fbc15dc60 2024-02-18T11:48:26.670 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.678+0000 7f3fb1ffb640 1 -- 172.21.15.116:0/4268438208 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f3fac0584b0 con 0x7f3fbc15dc60 2024-02-18T11:48:26.671 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:26.671 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:26.672 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3fa006bc70 msgr2=0x7f3fa006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3fa006bc70 0x7f3fa006e130 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8035000 tx=0x7f3fb802e000 comp rx=0 tx=0).stop 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 msgr2=0x7f3fbc15ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc15ed10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c420 tx=0x7f3fac00c8e0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 shutdown_connections 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3fa006bc70 0x7f3fa006e130 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fbc161850 0x7f3fbc160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fbc15dc60 0x7f3fbc15ed10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 --2- 172.21.15.116:0/4268438208 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3fbc0d1080 0x7f3fbc1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 >> 172.21.15.116:0/4268438208 conn(0x7f3fbc0b5910 msgr2=0x7f3fbc153d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 shutdown_connections 2024-02-18T11:48:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.680+0000 7f3fc32c3640 1 -- 172.21.15.116:0/4268438208 wait complete. 2024-02-18T11:48:26.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:26.684 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:26.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.795+0000 7f79b6d48640 1 Processor -- start 2024-02-18T11:48:26.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.795+0000 7f79b6d48640 1 -- start start 2024-02-18T11:48:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.795+0000 7f79b6d48640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f79b015dc90 0x7f79b00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.795+0000 7f79b6d48640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00cf980 con 0x7f79b006f300 2024-02-18T11:48:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.795+0000 7f79b6d48640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b006ab60 con 0x7f79b00cf3e0 2024-02-18T11:48:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.795+0000 7f79b6d48640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b015e070 con 0x7f79b015dc90 2024-02-18T11:48:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b4ae4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f79b015dc90 0x7f79b00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b4ae4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f79b015dc90 0x7f79b00ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45042/0 (socket says 172.21.15.116:45042) 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b52e5640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f79b006f300 0x7f79b00b2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53764/0 (socket says 172.21.15.116:53764) 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b4ae4640 1 -- 172.21.15.116:0/4197199493 learned_addr learned my addr 172.21.15.116:0/4197199493 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b4ae4640 1 -- 172.21.15.116:0/4197199493 >> v1:172.21.15.143:6789/0 conn(0x7f79b00cf3e0 legacy=0x7f79b006ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b4ae4640 1 -- 172.21.15.116:0/4197199493 >> v1:172.21.15.116:6789/0 conn(0x7f79b006f300 legacy=0x7f79b00b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79b4ae4640 1 -- 172.21.15.116:0/4197199493 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b006ae40 con 0x7f79b015dc90 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.796+0000 7f79af7fe640 1 -- 172.21.15.116:0/4197199493 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468475010 0 0) 0x7f79b00cf980 con 0x7f79b006f300 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b4ae4640 1 --2- 172.21.15.116:0/4197199493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b015dc90 0x7f79b00ceea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f79b015e1b0 tx=0x7f79a402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fad3f78ef8ff82e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79af7fe640 1 -- 172.21.15.116:0/4197199493 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4030840 con 0x7f79b015dc90 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79af7fe640 1 -- 172.21.15.116:0/4197199493 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a40309a0 con 0x7f79b015dc90 2024-02-18T11:48:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79af7fe640 1 -- 172.21.15.116:0/4197199493 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4030d30 con 0x7f79b015dc90 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 -- 172.21.15.116:0/4197199493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b015dc90 msgr2=0x7f79b00ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 --2- 172.21.15.116:0/4197199493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b015dc90 0x7f79b00ceea0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f79b015e1b0 tx=0x7f79a402b980 comp rx=0 tx=0).stop 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 -- 172.21.15.116:0/4197199493 shutdown_connections 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 --2- 172.21.15.116:0/4197199493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b015dc90 0x7f79b00ceea0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 -- 172.21.15.116:0/4197199493 >> 172.21.15.116:0/4197199493 conn(0x7f79b00cfd80 msgr2=0x7f79b00d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 -- 172.21.15.116:0/4197199493 shutdown_connections 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.797+0000 7f79b6d48640 1 -- 172.21.15.116:0/4197199493 wait complete. 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.798+0000 7f79b6d48640 1 Processor -- start 2024-02-18T11:48:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.798+0000 7f79b6d48640 1 -- start start 2024-02-18T11:48:26.795 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b6d48640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 0x7f79b0158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b6d48640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79b00cf3e0 0x7f79b00afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b6d48640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f79b015dc90 0x7f79b00aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b6d48640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01f48c0 con 0x7f79b006f300 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b6d48640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015ffa0 con 0x7f79b00cf3e0 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b6d48640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0160280 con 0x7f79b015dc90 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b4ae4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 0x7f79b0158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b4ae4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 0x7f79b0158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45054/0 (socket says 172.21.15.116:45054) 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.799+0000 7f79b4ae4640 1 -- 172.21.15.116:0/2534573015 learned_addr learned my addr 172.21.15.116:0/2534573015 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b4ae4640 1 -- 172.21.15.116:0/2534573015 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f79b015dc90 msgr2=0x7f79b00aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b4ae4640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f79b015dc90 0x7f79b00aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b4ae4640 1 -- 172.21.15.116:0/2534573015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79b00cf3e0 msgr2=0x7f79b00afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b4ae4640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79b00cf3e0 0x7f79b00afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b4ae4640 1 -- 172.21.15.116:0/2534573015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40094d0 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b4ae4640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 0x7f79b0158b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f79a4000c00 tx=0x7f79a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4039b40 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4039cc0 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4043630 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b0160560 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.800+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0159080 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.801+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b0067180 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.801+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f79a404c3b0 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.802+0000 7f79ad7fa640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f799406bca0 0x7f799406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.802+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f79a40ad110 con 0x7f79b006f300 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.802+0000 7f79affff640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f799406bca0 0x7f799406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:26.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.802+0000 7f79affff640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f799406bca0 0x7f799406e160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f79b0161340 tx=0x7f799c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:26.798 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.806+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79a4077560 con 0x7f79b006f300 2024-02-18T11:48:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.976+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79b0074520 con 0x7f79b006f300 2024-02-18T11:48:26.968 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.976+0000 7f79ad7fa640 1 -- 172.21.15.116:0/2534573015 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79a407ac20 con 0x7f79b006f300 2024-02-18T11:48:26.968 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:26.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f799406bca0 msgr2=0x7f799406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.970 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f799406bca0 0x7f799406e160 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f79b0161340 tx=0x7f799c0079b0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 msgr2=0x7f79b0158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 0x7f79b0158b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f79a4000c00 tx=0x7f79a4004730 comp rx=0 tx=0).stop 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 shutdown_connections 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f799406bca0 0x7f799406e160 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f79b015dc90 0x7f79b00aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79b00cf3e0 0x7f79b00afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 --2- 172.21.15.116:0/2534573015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79b006f300 0x7f79b0158b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 >> 172.21.15.116:0/2534573015 conn(0x7f79b00cfd80 msgr2=0x7f79b00b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 shutdown_connections 2024-02-18T11:48:26.971 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:26.978+0000 7f79b6d48640 1 -- 172.21.15.116:0/2534573015 wait complete. 2024-02-18T11:48:26.981 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:27.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.091+0000 7fbf291f4640 1 Processor -- start 2024-02-18T11:48:27.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.091+0000 7fbf291f4640 1 -- start start 2024-02-18T11:48:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf291f4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbf2406afe0 0x7fbf240cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf291f4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2415df10 con 0x7fbf2415db30 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf291f4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240caa80 con 0x7fbf2406ab60 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf291f4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2406fa90 con 0x7fbf2406afe0 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf22d9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbf2415db30 0x7fbf240b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53770/0 (socket says 172.21.15.116:53770) 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf22d9d640 1 -- 172.21.15.116:0/4205290111 learned_addr learned my addr 172.21.15.116:0/4205290111 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531473788 0 0) 0x7fbf2415df10 con 0x7fbf2415db30 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf0c003520 con 0x7fbf2415db30 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.092+0000 7fbf2359e640 1 --2- 172.21.15.116:0/4205290111 >> v2:172.21.15.143:3300/0 conn(0x7fbf2406afe0 0x7fbf240cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335314700 0 0) 0x7fbf0c003520 con 0x7fbf2415db30 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 >> v2:172.21.15.143:3300/0 conn(0x7fbf2406afe0 msgr2=0x7fbf240cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 --2- 172.21.15.116:0/4205290111 >> v2:172.21.15.143:3300/0 conn(0x7fbf2406afe0 0x7fbf240cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 >> v1:172.21.15.143:6789/0 conn(0x7fbf2406ab60 legacy=0x7fbf240caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2406fc70 con 0x7fbf2415db30 2024-02-18T11:48:27.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fbf140019f0 con 0x7fbf2415db30 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 >> v1:172.21.15.116:6789/0 conn(0x7fbf2415db30 legacy=0x7fbf240b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c003970 0x7fbf0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2406fa90 con 0x7fbf0c0047c0 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c00c020 con 0x7fbf0c003970 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c00c1e0 con 0x7fbf0c005180 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2259c640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf22d9d640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c003970 0x7fbf0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2359e640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2259c640 1 -- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf0c005180 msgr2=0x7fbf0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2259c640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2259c640 1 -- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c003970 msgr2=0x7fbf0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2259c640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf0c003970 0x7fbf0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2259c640 1 -- 172.21.15.116:0/4205290111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c003520 con 0x7fbf0c0047c0 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2359e640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf0c005180 0x7fbf0c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.093+0000 7fbf2359e640 1 -- 172.21.15.116:0/4205290111 reap_dead start 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf2259c640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbf18001750 tx=0x7fbf0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa88c1d146e13a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180052d0 con 0x7fbf0c0047c0 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1802e050 con 0x7fbf0c0047c0 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf21d9b640 1 -- 172.21.15.116:0/4205290111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18003830 con 0x7fbf0c0047c0 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf291f4640 1 -- 172.21.15.116:0/4205290111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 msgr2=0x7fbf0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf291f4640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbf18001750 tx=0x7fbf0c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf291f4640 1 -- 172.21.15.116:0/4205290111 shutdown_connections 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf291f4640 1 --2- 172.21.15.116:0/4205290111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf0c004c40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.094+0000 7fbf291f4640 1 -- 172.21.15.116:0/4205290111 >> 172.21.15.116:0/4205290111 conn(0x7fbf240b23f0 msgr2=0x7fbf240b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.095+0000 7fbf291f4640 1 -- 172.21.15.116:0/4205290111 shutdown_connections 2024-02-18T11:48:27.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.095+0000 7fbf291f4640 1 -- 172.21.15.116:0/4205290111 wait complete. 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.096+0000 7fbf291f4640 1 Processor -- start 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.096+0000 7fbf291f4640 1 -- start start 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf24147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf1b0 0x7fbf24147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf240cf630 0x7fbf24148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24156f50 con 0x7fbf0c0047c0 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ced20 con 0x7fbf240cf1b0 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24147850 con 0x7fbf240cf630 2024-02-18T11:48:27.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf24147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf24147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45084/0 (socket says 172.21.15.116:45084) 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 -- 172.21.15.116:0/2627061324 learned_addr learned my addr 172.21.15.116:0/2627061324 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 -- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf240cf630 msgr2=0x7fbf24148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf240cf630 0x7fbf24148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 -- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf1b0 msgr2=0x7fbf24147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf2259c640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf1b0 0x7fbf24147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf1b0 0x7fbf24147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 -- 172.21.15.116:0/2627061324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf240ce110 con 0x7fbf0c0047c0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf2259c640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf1b0 0x7fbf24147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf22d9d640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf24147310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf14007970 tx=0x7fbf1400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400ad10 con 0x7fbf0c0047c0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400ae70 con 0x7fbf0c0047c0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240ce3f0 con 0x7fbf0c0047c0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.097+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14013610 con 0x7fbf0c0047c0 2024-02-18T11:48:27.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.098+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf240ce9b0 con 0x7fbf0c0047c0 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.099+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf240ceb40 con 0x7fbf0c0047c0 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.099+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fbf14013770 con 0x7fbf0c0047c0 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.099+0000 7fbf0b7fe640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbf1006bc70 0x7fbf1006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.099+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fbf1408b770 con 0x7fbf0c0047c0 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.099+0000 7fbf2259c640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbf1006bc70 0x7fbf1006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.100+0000 7fbf2259c640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbf1006bc70 0x7fbf1006e130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf18033000 tx=0x7fbf18009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.104+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf14055be0 con 0x7fbf0c0047c0 2024-02-18T11:48:27.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.272+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf24074520 con 0x7fbf0c0047c0 2024-02-18T11:48:27.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.272+0000 7fbf0b7fe640 1 -- 172.21.15.116:0/2627061324 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf140592a0 con 0x7fbf0c0047c0 2024-02-18T11:48:27.265 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbf1006bc70 msgr2=0x7fbf1006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbf1006bc70 0x7fbf1006e130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf18033000 tx=0x7fbf18009030 comp rx=0 tx=0).stop 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 msgr2=0x7fbf24147310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf24147310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbf14007970 tx=0x7fbf1400a6e0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 shutdown_connections 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fbf1006bc70 0x7fbf1006e130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbf240cf630 0x7fbf24148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf240cf1b0 0x7fbf24147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 --2- 172.21.15.116:0/2627061324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf0c0047c0 0x7fbf24147310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 >> 172.21.15.116:0/2627061324 conn(0x7fbf240b23f0 msgr2=0x7fbf2415b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 shutdown_connections 2024-02-18T11:48:27.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.274+0000 7fbf291f4640 1 -- 172.21.15.116:0/2627061324 wait complete. 2024-02-18T11:48:27.277 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.388+0000 7f8726c64640 1 Processor -- start 2024-02-18T11:48:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.388+0000 7f8726c64640 1 -- start start 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8726c64640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87200cdd20 0x7f8720071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8726c64640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8720071f90 0x7f872015da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8726c64640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872015a480 con 0x7f8720150ed0 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8726c64640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8720072420 con 0x7f87200cdd20 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8726c64640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f872015a760 con 0x7f8720071f90 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f871ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87200cdd20 0x7f8720071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f871ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87200cdd20 0x7f8720071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45100/0 (socket says 172.21.15.116:45100) 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f871ffff640 1 -- 172.21.15.116:0/642111616 learned_addr learned my addr 172.21.15.116:0/642111616 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8724a00640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8720150ed0 0x7f87200b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53772/0 (socket says 172.21.15.116:53772) 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.389+0000 7f8725201640 1 --2- 172.21.15.116:0/642111616 >> v2:172.21.15.143:3300/0 conn(0x7f8720071f90 0x7f872015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871ffff640 1 -- 172.21.15.116:0/642111616 >> v2:172.21.15.143:3300/0 conn(0x7f8720071f90 msgr2=0x7f872015da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871ffff640 1 --2- 172.21.15.116:0/642111616 >> v2:172.21.15.143:3300/0 conn(0x7f8720071f90 0x7f872015da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871ffff640 1 -- 172.21.15.116:0/642111616 >> v1:172.21.15.116:6789/0 conn(0x7f8720150ed0 legacy=0x7f87200b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871ffff640 1 -- 172.21.15.116:0/642111616 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872015aa40 con 0x7f87200cdd20 2024-02-18T11:48:27.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871f7fe640 1 -- 172.21.15.116:0/642111616 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259698141 0 0) 0x7f872015a480 con 0x7f8720150ed0 2024-02-18T11:48:27.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871ffff640 1 --2- 172.21.15.116:0/642111616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720071a50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f871400bb10 tx=0x7f871402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4c6fc9574f289f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871f7fe640 1 -- 172.21.15.116:0/642111616 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871402ee90 con 0x7f87200cdd20 2024-02-18T11:48:27.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871f7fe640 1 -- 172.21.15.116:0/642111616 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714036440 con 0x7f87200cdd20 2024-02-18T11:48:27.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.390+0000 7f871f7fe640 1 -- 172.21.15.116:0/642111616 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87140367d0 con 0x7f87200cdd20 2024-02-18T11:48:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 -- 172.21.15.116:0/642111616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 msgr2=0x7f8720071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 --2- 172.21.15.116:0/642111616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720071a50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f871400bb10 tx=0x7f871402e0e0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 -- 172.21.15.116:0/642111616 shutdown_connections 2024-02-18T11:48:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 --2- 172.21.15.116:0/642111616 >> v2:172.21.15.143:3300/0 conn(0x7f8720071f90 0x7f872015da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 --2- 172.21.15.116:0/642111616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720071a50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 -- 172.21.15.116:0/642111616 >> 172.21.15.116:0/642111616 conn(0x7f8720067000 msgr2=0x7f872006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 -- 172.21.15.116:0/642111616 shutdown_connections 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.391+0000 7f8726c64640 1 -- 172.21.15.116:0/642111616 wait complete. 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.392+0000 7f8726c64640 1 Processor -- start 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.392+0000 7f8726c64640 1 -- start start 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8726c64640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8720071f90 0x7f8720151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8726c64640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8726c64640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8720150ed0 0x7f87200711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8726c64640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8720147590 con 0x7f87200cdd20 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8726c64640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87201f7870 con 0x7f8720150ed0 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8726c64640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87201f6cf0 con 0x7f8720071f90 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45108/0 (socket says 172.21.15.116:45108) 2024-02-18T11:48:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 -- 172.21.15.116:0/4255545541 learned_addr learned my addr 172.21.15.116:0/4255545541 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8724a00640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8720071f90 0x7f8720151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f8725201640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8720150ed0 0x7f87200711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 -- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8720071f90 msgr2=0x7f8720151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8720071f90 0x7f8720151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 -- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8720150ed0 msgr2=0x7f87200711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8720150ed0 0x7f87200711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.393+0000 7f871ffff640 1 -- 172.21.15.116:0/4255545541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871400b770 con 0x7f87200cdd20 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.394+0000 7f871ffff640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720152670 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f871402e120 tx=0x7f87140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.394+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87140348d0 con 0x7f87200cdd20 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.394+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714034a50 con 0x7f87200cdd20 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.394+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87201f6fd0 con 0x7f87200cdd20 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.394+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714034e00 con 0x7f87200cdd20 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.394+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87201f7590 con 0x7f87200cdd20 2024-02-18T11:48:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.395+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87200ac950 con 0x7f87200cdd20 2024-02-18T11:48:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.395+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f871404a3b0 con 0x7f87200cdd20 2024-02-18T11:48:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.395+0000 7f871d7fa640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f870c06bca0 0x7f870c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.396+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f87140ad0f0 con 0x7f87200cdd20 2024-02-18T11:48:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.396+0000 7f8724a00640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f870c06bca0 0x7f870c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.396+0000 7f8724a00640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f870c06bca0 0x7f870c06e160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87100012e0 tx=0x7f8710007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.400+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87200ac950 con 0x7f87200cdd20 2024-02-18T11:48:27.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.569+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87200ac950 con 0x7f87200cdd20 2024-02-18T11:48:27.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.569+0000 7f871d7fa640 1 -- 172.21.15.116:0/4255545541 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87200ac950 con 0x7f87200cdd20 2024-02-18T11:48:27.562 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:27.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f870c06bca0 msgr2=0x7f870c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f870c06bca0 0x7f870c06e160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f87100012e0 tx=0x7f8710007ed0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 msgr2=0x7f8720152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720152670 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f871402e120 tx=0x7f87140046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 shutdown_connections 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f870c06bca0 0x7f870c06e160 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8720150ed0 0x7f87200711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87200cdd20 0x7f8720152670 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 --2- 172.21.15.116:0/4255545541 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8720071f90 0x7f8720151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 >> 172.21.15.116:0/4255545541 conn(0x7f8720067000 msgr2=0x7f87200b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 shutdown_connections 2024-02-18T11:48:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.571+0000 7f8726c64640 1 -- 172.21.15.116:0/4255545541 wait complete. 2024-02-18T11:48:27.574 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:27.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.685+0000 7f97b0b77640 1 Processor -- start 2024-02-18T11:48:27.677 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.685+0000 7f97b0b77640 1 -- start start 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97b0b77640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f97ac0cdd20 0x7f97ac071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97b0b77640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac15da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97b0b77640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac15a480 con 0x7f97ac150ed0 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97b0b77640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac072420 con 0x7f97ac0cdd20 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97b0b77640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac15a760 con 0x7f97ac071f90 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97a9d9b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f97ac0cdd20 0x7f97ac071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac15da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:39204/0 (socket says 172.21.15.116:39204) 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.686+0000 7f97aad9d640 1 -- 172.21.15.116:0/3784266355 learned_addr learned my addr 172.21.15.116:0/3784266355 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.687+0000 7f97a9d9b640 1 -- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 msgr2=0x7f97ac15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.687+0000 7f97a9d9b640 1 --2- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.687+0000 7f97a9d9b640 1 -- 172.21.15.116:0/3784266355 >> v1:172.21.15.143:6789/0 conn(0x7f97ac150ed0 legacy=0x7f97ac0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.680 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.687+0000 7f97a9d9b640 1 -- 172.21.15.116:0/3784266355 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac15aa40 con 0x7f97ac0cdd20 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.687+0000 7f97a9d9b640 1 --2- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 0x7f97ac071a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097d0 tx=0x7f97a002c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b600bd02fcf89ea server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97a959a640 1 -- 172.21.15.116:0/3784266355 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0032880 con 0x7f97ac0cdd20 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97a959a640 1 -- 172.21.15.116:0/3784266355 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00329e0 con 0x7f97ac0cdd20 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97a959a640 1 -- 172.21.15.116:0/3784266355 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0032d70 con 0x7f97ac0cdd20 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 -- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 msgr2=0x7f97ac071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 --2- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 0x7f97ac071a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97a00097d0 tx=0x7f97a002c920 comp rx=0 tx=0).stop 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 -- 172.21.15.116:0/3784266355 shutdown_connections 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 --2- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 --2- 172.21.15.116:0/3784266355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 0x7f97ac071a50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.681 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 -- 172.21.15.116:0/3784266355 >> 172.21.15.116:0/3784266355 conn(0x7f97ac067000 msgr2=0x7f97ac06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 -- 172.21.15.116:0/3784266355 shutdown_connections 2024-02-18T11:48:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.688+0000 7f97b0b77640 1 -- 172.21.15.116:0/3784266355 wait complete. 2024-02-18T11:48:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.689+0000 7f97b0b77640 1 Processor -- start 2024-02-18T11:48:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 -- start start 2024-02-18T11:48:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac1f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 0x7f97ac1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 0x7f97ac144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac155980 con 0x7f97ac150ed0 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac159620 con 0x7f97ac0cdd20 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97b0b77640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1598d0 con 0x7f97ac071f90 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 0x7f97ac144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 0x7f97ac144df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34964/0 (socket says 172.21.15.116:34964) 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.690+0000 7f97aad9d640 1 -- 172.21.15.116:0/2474677983 learned_addr learned my addr 172.21.15.116:0/2474677983 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aa59c640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac1f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aad9d640 1 -- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 msgr2=0x7f97ac1f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aad9d640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac1f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aad9d640 1 -- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 msgr2=0x7f97ac1590e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aad9d640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 0x7f97ac1590e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aad9d640 1 -- 172.21.15.116:0/2474677983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009400 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aa59c640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac1f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97aad9d640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 0x7f97ac144df0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f97a400e870 tx=0x7f97a400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4009660 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400c940 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac159bb0 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400ccf0 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.691+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac155bd0 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.692+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac074520 con 0x7f97ac150ed0 2024-02-18T11:48:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.692+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f97a4013470 con 0x7f97ac150ed0 2024-02-18T11:48:27.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.693+0000 7f979affd640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f978c06bca0 0x7f978c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.693+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f97a408ae30 con 0x7f97ac150ed0 2024-02-18T11:48:27.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.693+0000 7f97aa59c640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f978c06bca0 0x7f978c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.693+0000 7f97aa59c640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f978c06bca0 0x7f978c06e160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f979c001cd0 tx=0x7f979c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.689 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.697+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97a4055170 con 0x7f97ac150ed0 2024-02-18T11:48:27.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.867+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f97ac06f150 con 0x7f97ac150ed0 2024-02-18T11:48:27.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.867+0000 7f979affd640 1 -- 172.21.15.116:0/2474677983 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f97a4058830 con 0x7f97ac150ed0 2024-02-18T11:48:27.860 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f978c06bca0 msgr2=0x7f978c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f978c06bca0 0x7f978c06e160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f979c001cd0 tx=0x7f979c009040 comp rx=0 tx=0).stop 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 msgr2=0x7f97ac144df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 0x7f97ac144df0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f97a400e870 tx=0x7f97a400ed30 comp rx=0 tx=0).stop 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 shutdown_connections 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f978c06bca0 0x7f978c06e160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97ac150ed0 0x7f97ac144df0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97ac0cdd20 0x7f97ac1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 --2- 172.21.15.116:0/2474677983 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f97ac071f90 0x7f97ac1f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 >> 172.21.15.116:0/2474677983 conn(0x7f97ac067000 msgr2=0x7f97ac14ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 shutdown_connections 2024-02-18T11:48:27.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.869+0000 7f97b0b77640 1 -- 172.21.15.116:0/2474677983 wait complete. 2024-02-18T11:48:27.872 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:48:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.980+0000 7fe09b9d7640 1 Processor -- start 2024-02-18T11:48:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.981+0000 7fe09b9d7640 1 -- start start 2024-02-18T11:48:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.981+0000 7fe09b9d7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe09415dbf0 0x7fe0941617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.981+0000 7fe09b9d7640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.981+0000 7fe09b9d7640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0940d1ee0 con 0x7fe0940b9370 2024-02-18T11:48:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.981+0000 7fe09b9d7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe094161dc0 con 0x7fe09415dbf0 2024-02-18T11:48:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.981+0000 7fe09b9d7640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0941620a0 con 0x7fe0940cee50 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099773640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe09415dbf0 0x7fe0941617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099f74640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe098f72640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe0940b9370 0x7fe0940ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:56050/0 (socket says 172.21.15.116:56050) 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe098f72640 1 -- 172.21.15.116:0/429817141 learned_addr learned my addr 172.21.15.116:0/429817141 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099f74640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:39222/0 (socket says 172.21.15.116:39222) 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099773640 1 -- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 msgr2=0x7fe0940d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099773640 1 --2- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099773640 1 -- 172.21.15.116:0/429817141 >> v1:172.21.15.143:6789/0 conn(0x7fe0940b9370 legacy=0x7fe0940ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099773640 1 -- 172.21.15.116:0/429817141 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe094162380 con 0x7fe09415dbf0 2024-02-18T11:48:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.982+0000 7fe099f74640 1 --2- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.983+0000 7fe099773640 1 --2- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0941617e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097a0 tx=0x7fe08402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98bfc191c77e126e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.983+0000 7fe08bfff640 1 -- 172.21.15.116:0/429817141 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084031830 con 0x7fe09415dbf0 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.983+0000 7fe08bfff640 1 -- 172.21.15.116:0/429817141 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe084031990 con 0x7fe09415dbf0 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.983+0000 7fe08bfff640 1 -- 172.21.15.116:0/429817141 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084031d20 con 0x7fe09415dbf0 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 -- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 msgr2=0x7fe0941617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0941617e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097a0 tx=0x7fe08402b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 -- 172.21.15.116:0/429817141 shutdown_connections 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/429817141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0941617e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 -- 172.21.15.116:0/429817141 >> 172.21.15.116:0/429817141 conn(0x7fe0940b5480 msgr2=0x7fe0940b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 -- 172.21.15.116:0/429817141 shutdown_connections 2024-02-18T11:48:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.984+0000 7fe09b9d7640 1 -- 172.21.15.116:0/429817141 wait complete. 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.985+0000 7fe09b9d7640 1 Processor -- start 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.985+0000 7fe09b9d7640 1 -- start start 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe09b9d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 0x7fe09415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe09b9d7640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe09b9d7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0940c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe09b9d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094159b90 con 0x7fe0940b9370 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe09b9d7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094159e70 con 0x7fe09415dbf0 2024-02-18T11:48:27.979 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe09b9d7640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09415a150 con 0x7fe0940cee50 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 0x7fe09415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 0x7fe09415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34972/0 (socket says 172.21.15.116:34972) 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 -- 172.21.15.116:0/3815264755 learned_addr learned my addr 172.21.15.116:0/3815264755 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe098f72640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099f74640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0940c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 -- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 msgr2=0x7fe0940aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 -- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 msgr2=0x7fe0940c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0940c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 -- 172.21.15.116:0/3815264755 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c009a70 con 0x7fe0940b9370 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe099773640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 0x7fe09415b470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe08402bee0 tx=0x7fe0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.986+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08403a6f0 con 0x7fe0940b9370 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.987+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08403a870 con 0x7fe0940b9370 2024-02-18T11:48:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.987+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe084009400 con 0x7fe0940b9370 2024-02-18T11:48:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.987+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08403ac20 con 0x7fe0940b9370 2024-02-18T11:48:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.987+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09415e150 con 0x7fe0940b9370 2024-02-18T11:48:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.988+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe094067840 con 0x7fe0940b9370 2024-02-18T11:48:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.988+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe084030380 con 0x7fe0940b9370 2024-02-18T11:48:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.988+0000 7fe089ffb640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe07806bc70 0x7fe07806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.988+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe0840ac560 con 0x7fe0940b9370 2024-02-18T11:48:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.988+0000 7fe098f72640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe07806bc70 0x7fe07806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.989+0000 7fe098f72640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe07806bc70 0x7fe07806e130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0940b77a0 tx=0x7fe090009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:27.985 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:27.993+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe084076930 con 0x7fe0940b9370 2024-02-18T11:48:28.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.176+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe094159cd0 con 0x7fe0940b9370 2024-02-18T11:48:28.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.177+0000 7fe089ffb640 1 -- 172.21.15.116:0/3815264755 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe094159cd0 con 0x7fe0940b9370 2024-02-18T11:48:28.170 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T11:48:28.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe07806bc70 msgr2=0x7fe07806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe07806bc70 0x7fe07806e130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe0940b77a0 tx=0x7fe090009040 comp rx=0 tx=0).stop 2024-02-18T11:48:28.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 msgr2=0x7fe09415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 0x7fe09415b470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe08402bee0 tx=0x7fe0840046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 shutdown_connections 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe07806bc70 0x7fe07806e130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09415dbf0 0x7fe0940c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0940cee50 0x7fe0940aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 --2- 172.21.15.116:0/3815264755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0940b9370 0x7fe09415b470 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.178+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 >> 172.21.15.116:0/3815264755 conn(0x7fe0940b5480 msgr2=0x7fe0940b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:28.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.179+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 shutdown_connections 2024-02-18T11:48:28.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.179+0000 7fe09b9d7640 1 -- 172.21.15.116:0/3815264755 wait complete. 2024-02-18T11:48:28.181 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-02-18T11:48:28.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:28.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-02-18T11:48:28.216 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:48:28.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:28.217 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-18T11:48:28.284 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:28.284 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:28.300 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:28.301 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:28.356 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:28.356 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:28.411 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.559+0000 7f8510c0f640 1 Processor -- start 2024-02-18T11:48:28.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.559+0000 7f8510c0f640 1 -- start start 2024-02-18T11:48:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f8510c0f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 0x7f850c0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f8510c0f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c0af870 con 0x7f850c0ad570 2024-02-18T11:48:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f8510c0f640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c15c530 con 0x7f850c06c1f0 2024-02-18T11:48:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f8510c0f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c06c5d0 con 0x7f850c15c9c0 2024-02-18T11:48:28.554 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f850ad9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f850c0ad570 0x7f850c06ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55566/0 (socket says 172.21.15.116:55566) 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f850ad9d640 1 -- 172.21.15.116:0/3772240250 learned_addr learned my addr 172.21.15.116:0/3772240250 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f8509d9b640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 0x7f850c0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003289846 0 0) 0x7f850c0af870 con 0x7f850c0ad570 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.560+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84f4003520 con 0x7f850c0ad570 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1232565556 0 0) 0x7f84f4003520 con 0x7f850c0ad570 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 msgr2=0x7f850c0ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 0x7f850c0ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 >> v1:172.21.15.143:6789/0 conn(0x7f850c06c1f0 legacy=0x7f850c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0af590 con 0x7f850c0ad570 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f85040019f0 con 0x7f850c0ad570 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 >> v1:172.21.15.116:6789/0 conn(0x7f850c0ad570 legacy=0x7f850c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f84f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f40047c0 0x7f84f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 0x7f84f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c06c5d0 con 0x7f84f40039a0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f400bab0 con 0x7f84f40047c0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f400bc70 con 0x7f850c15c9c0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850a59c640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f84f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f8509d9b640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f40047c0 0x7f84f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850ad9d640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 0x7f84f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850a59c640 1 -- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 msgr2=0x7f84f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850a59c640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15c9c0 0x7f84f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850a59c640 1 -- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f40047c0 msgr2=0x7f84f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850a59c640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84f40047c0 0x7f84f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.561+0000 7f850a59c640 1 -- 172.21.15.116:0/3772240250 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f4003520 con 0x7f84f40039a0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.562+0000 7f850a59c640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f84f4003dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001750 tx=0x7f84f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57b39b3042914ae server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.562+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0052d0 con 0x7f84f40039a0 2024-02-18T11:48:28.556 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.562+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc02e050 con 0x7f84f40039a0 2024-02-18T11:48:28.557 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.562+0000 7f850959a640 1 -- 172.21.15.116:0/3772240250 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0035f0 con 0x7f84f40039a0 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.562+0000 7f8510c0f640 1 -- 172.21.15.116:0/3772240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 msgr2=0x7f84f4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.563+0000 7f8510c0f640 1 --2- 172.21.15.116:0/3772240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f84f4003dc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001750 tx=0x7f84f40057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.563+0000 7f850ad9d640 1 -- 172.21.15.116:0/3772240250 reap_dead start 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.563+0000 7f8510c0f640 1 -- 172.21.15.116:0/3772240250 shutdown_connections 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.563+0000 7f8510c0f640 1 -- 172.21.15.116:0/3772240250 >> 172.21.15.116:0/3772240250 conn(0x7f850c0b23d0 msgr2=0x7f850c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.563+0000 7f8510c0f640 1 -- 172.21.15.116:0/3772240250 shutdown_connections 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.563+0000 7f8510c0f640 1 -- 172.21.15.116:0/3772240250 wait complete. 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.564+0000 7f8510c0f640 1 Processor -- start 2024-02-18T11:48:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.564+0000 7f8510c0f640 1 -- start start 2024-02-18T11:48:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f850c152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f850c14e9d0 0x7f850c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15a9b0 0x7f850c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c14c900 con 0x7f84f40039a0 2024-02-18T11:48:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c15b3a0 con 0x7f850c14e9d0 2024-02-18T11:48:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c159570 con 0x7f850c15a9b0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f850c152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f850c152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34986/0 (socket says 172.21.15.116:34986) 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 -- 172.21.15.116:0/449007552 learned_addr learned my addr 172.21.15.116:0/449007552 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 -- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15a9b0 msgr2=0x7f850c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15a9b0 0x7f850c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 -- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f850c14e9d0 msgr2=0x7f850c1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8509d9b640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f850c14e9d0 0x7f850c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f850c14e9d0 0x7f850c1572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 -- 172.21.15.116:0/449007552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c158d50 con 0x7f84f40039a0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f850a59c640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f850c152040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f84fc008fd0 tx=0x7f84fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0076c0 con 0x7f84f40039a0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc007a30 con 0x7f84f40039a0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc006c20 con 0x7f84f40039a0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c158ff0 con 0x7f84f40039a0 2024-02-18T11:48:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.565+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c14c520 con 0x7f84f40039a0 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.566+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c074520 con 0x7f84f40039a0 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.567+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f84fc0035f0 con 0x7f84f40039a0 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.567+0000 7f84faffd640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f84ec06bca0 0x7f84ec06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.567+0000 7f8509d9b640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f84ec06bca0 0x7f84ec06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.567+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f84fc0ac780 con 0x7f84f40039a0 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.567+0000 7f8509d9b640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f84ec06bca0 0x7f84ec06e160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f850c1f5ab0 tx=0x7f8500008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.571+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84fc076bd0 con 0x7f84f40039a0 2024-02-18T11:48:28.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.741+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f850c0d09c0 con 0x7f84f40039a0 2024-02-18T11:48:28.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.742+0000 7f84faffd640 1 -- 172.21.15.116:0/449007552 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f84fc07a290 con 0x7f84f40039a0 2024-02-18T11:48:28.735 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:28.735 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f84ec06bca0 msgr2=0x7f84ec06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f84ec06bca0 0x7f84ec06e160 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f850c1f5ab0 tx=0x7f8500008040 comp rx=0 tx=0).stop 2024-02-18T11:48:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 msgr2=0x7f850c152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f850c152040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f84fc008fd0 tx=0x7f84fc001120 comp rx=0 tx=0).stop 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 shutdown_connections 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f84ec06bca0 0x7f84ec06e160 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f850c15a9b0 0x7f850c15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f850c14e9d0 0x7f850c1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 --2- 172.21.15.116:0/449007552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84f40039a0 0x7f850c152040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.744+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 >> 172.21.15.116:0/449007552 conn(0x7f850c0b23d0 msgr2=0x7f850c155560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.745+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 shutdown_connections 2024-02-18T11:48:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.745+0000 7f8510c0f640 1 -- 172.21.15.116:0/449007552 wait complete. 2024-02-18T11:48:28.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:28.748 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:28.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.859+0000 7f10d7b4b640 1 Processor -- start 2024-02-18T11:48:28.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.859+0000 7f10d7b4b640 1 -- start start 2024-02-18T11:48:28.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d7b4b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10d015dbf0 0x7f10d01617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d7b4b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d00d1ee0 con 0x7f10d00cee50 2024-02-18T11:48:28.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d7b4b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d0161dc0 con 0x7f10d00b9370 2024-02-18T11:48:28.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d7b4b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d01620a0 con 0x7f10d015dbf0 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d58e7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10d015dbf0 0x7f10d01617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d60e8640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f10d00cee50 0x7f10d00d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55578/0 (socket says 172.21.15.116:55578) 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d60e8640 1 -- 172.21.15.116:0/1408454795 learned_addr learned my addr 172.21.15.116:0/1408454795 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d58e7640 1 -- 172.21.15.116:0/1408454795 >> v1:172.21.15.143:6789/0 conn(0x7f10d00b9370 legacy=0x7f10d00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d58e7640 1 -- 172.21.15.116:0/1408454795 >> v1:172.21.15.116:6789/0 conn(0x7f10d00cee50 legacy=0x7f10d00d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d58e7640 1 -- 172.21.15.116:0/1408454795 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0162380 con 0x7f10d015dbf0 2024-02-18T11:48:28.853 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.860+0000 7f10d48e5640 1 -- 172.21.15.116:0/1408454795 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934211832 0 0) 0x7f10d00d1ee0 con 0x7f10d00cee50 2024-02-18T11:48:28.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d58e7640 1 --2- 172.21.15.116:0/1408454795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d015dbf0 0x7f10d01617e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0097a0 tx=0x7f10cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a7989b13c8d61 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:28.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d48e5640 1 -- 172.21.15.116:0/1408454795 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc031830 con 0x7f10d015dbf0 2024-02-18T11:48:28.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d48e5640 1 -- 172.21.15.116:0/1408454795 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc031990 con 0x7f10d015dbf0 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d48e5640 1 -- 172.21.15.116:0/1408454795 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc031d20 con 0x7f10d015dbf0 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d7b4b640 1 -- 172.21.15.116:0/1408454795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d015dbf0 msgr2=0x7f10d01617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/1408454795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d015dbf0 0x7f10d01617e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0097a0 tx=0x7f10cc02b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d7b4b640 1 -- 172.21.15.116:0/1408454795 shutdown_connections 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/1408454795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d015dbf0 0x7f10d01617e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.861+0000 7f10d7b4b640 1 -- 172.21.15.116:0/1408454795 >> 172.21.15.116:0/1408454795 conn(0x7f10d00b5480 msgr2=0x7f10d00b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.862+0000 7f10d7b4b640 1 -- 172.21.15.116:0/1408454795 shutdown_connections 2024-02-18T11:48:28.855 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.862+0000 7f10d7b4b640 1 -- 172.21.15.116:0/1408454795 wait complete. 2024-02-18T11:48:28.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 Processor -- start 2024-02-18T11:48:28.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 -- start start 2024-02-18T11:48:28.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 0x7f10d015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.858 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f10d00cee50 0x7f10d00d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d015dbf0 0x7f10d00d3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d015c790 con 0x7f10d00b9370 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d015a4d0 con 0x7f10d015dbf0 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d7b4b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0159ba0 con 0x7f10d00cee50 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d58e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 0x7f10d015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d58e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 0x7f10d015c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35004/0 (socket says 172.21.15.116:35004) 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.863+0000 7f10d58e7640 1 -- 172.21.15.116:0/109531656 learned_addr learned my addr 172.21.15.116:0/109531656 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d58e7640 1 -- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f10d00cee50 msgr2=0x7f10d00d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d50e6640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f10d00cee50 0x7f10d00d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d58e7640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f10d00cee50 0x7f10d00d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.859 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d60e8640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d015dbf0 0x7f10d00d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d58e7640 1 -- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d015dbf0 msgr2=0x7f10d00d3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d58e7640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d015dbf0 0x7f10d00d3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d58e7640 1 -- 172.21.15.116:0/109531656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc009400 con 0x7f10d00b9370 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d50e6640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f10d00cee50 0x7f10d00d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d60e8640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d015dbf0 0x7f10d00d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d58e7640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 0x7f10d015c0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10cc034040 tx=0x7f10cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0317d0 con 0x7f10d00b9370 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc004030 con 0x7f10d00b9370 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0043e0 con 0x7f10d00b9370 2024-02-18T11:48:28.860 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d015e150 con 0x7f10d00b9370 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.864+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d015a160 con 0x7f10d00b9370 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.865+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d0067840 con 0x7f10d00b9370 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.866+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f10cc043070 con 0x7f10d00b9370 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.866+0000 7f10c67fc640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f10b406bc70 0x7f10b406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.866+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f10cc0ac9c0 con 0x7f10d00b9370 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.866+0000 7f10d50e6640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f10b406bc70 0x7f10b406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:28.861 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.866+0000 7f10d50e6640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f10b406bc70 0x7f10b406e130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007fd0 tx=0x7f10c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:28.862 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:28.871+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10cc076d80 con 0x7f10d00b9370 2024-02-18T11:48:29.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.039+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10d0074520 con 0x7f10d00b9370 2024-02-18T11:48:29.032 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.040+0000 7f10c67fc640 1 -- 172.21.15.116:0/109531656 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10cc07a440 con 0x7f10d00b9370 2024-02-18T11:48:29.032 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:29.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f10b406bc70 msgr2=0x7f10b406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f10b406bc70 0x7f10b406e130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10c0007fd0 tx=0x7f10c0009040 comp rx=0 tx=0).stop 2024-02-18T11:48:29.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 msgr2=0x7f10d015c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 0x7f10d015c0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10cc034040 tx=0x7f10cc0046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 shutdown_connections 2024-02-18T11:48:29.034 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f10b406bc70 0x7f10b406e130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10d015dbf0 0x7f10d00d3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f10d00cee50 0x7f10d00d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 --2- 172.21.15.116:0/109531656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10d00b9370 0x7f10d015c0f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 >> 172.21.15.116:0/109531656 conn(0x7f10d00b5480 msgr2=0x7f10d00b7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 shutdown_connections 2024-02-18T11:48:29.035 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.042+0000 7f10d7b4b640 1 -- 172.21.15.116:0/109531656 wait complete. 2024-02-18T11:48:29.045 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:29.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.155+0000 7ffa3364e640 1 Processor -- start 2024-02-18T11:48:29.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.155+0000 7ffa3364e640 1 -- start start 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa3364e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffa2c1617e0 0x7ffa2c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa3364e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa2c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa3364e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c0d1ee0 con 0x7ffa2c15dbf0 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa3364e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c0cf110 con 0x7ffa2c1617e0 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa3364e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c161dc0 con 0x7ffa2c0cec80 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa313ea640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffa2c15dbf0 0x7ffa2c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55582/0 (socket says 172.21.15.116:55582) 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.156+0000 7ffa313ea640 1 -- 172.21.15.116:0/3270731005 learned_addr learned my addr 172.21.15.116:0/3270731005 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa31beb640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa2c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774121936 0 0) 0x7ffa2c0d1ee0 con 0x7ffa2c15dbf0 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 --2- 172.21.15.116:0/3270731005 >> v2:172.21.15.143:3300/0 conn(0x7ffa2c1617e0 0x7ffa2c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa18003520 con 0x7ffa2c15dbf0 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046350240 0 0) 0x7ffa18003520 con 0x7ffa2c15dbf0 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 msgr2=0x7ffa2c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa2c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 >> v2:172.21.15.143:3300/0 conn(0x7ffa2c1617e0 msgr2=0x7ffa2c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 --2- 172.21.15.116:0/3270731005 >> v2:172.21.15.143:3300/0 conn(0x7ffa2c1617e0 0x7ffa2c0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c1620a0 con 0x7ffa2c15dbf0 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7ffa28001670 con 0x7ffa2c15dbf0 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa31beb640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa2c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 >> v1:172.21.15.116:6789/0 conn(0x7ffa2c15dbf0 legacy=0x7ffa2c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa18003970 0x7ffa18003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa180047c0 0x7ffa18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0cf110 con 0x7ffa180047c0 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c161dc0 con 0x7ffa18003970 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1800ba00 con 0x7ffa2c0cec80 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa180047c0 0x7ffa18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa313ea640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa18003970 0x7ffa18003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa31beb640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 -- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 msgr2=0x7ffa18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0cec80 0x7ffa18005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 -- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa18003970 msgr2=0x7ffa18003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa18003970 0x7ffa18003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa30be9640 1 -- 172.21.15.116:0/3270731005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa18003520 con 0x7ffa180047c0 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.157+0000 7ffa313ea640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa18003970 0x7ffa18003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.158+0000 7ffa30be9640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa180047c0 0x7ffa18004c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009ac0 tx=0x7ffa180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3dc0690f52b94a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.158+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c02c710 con 0x7ffa180047c0 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.158+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c02c870 con 0x7ffa180047c0 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.158+0000 7ffa23fff640 1 -- 172.21.15.116:0/3270731005 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c02cc00 con 0x7ffa180047c0 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa3364e640 1 -- 172.21.15.116:0/3270731005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa180047c0 msgr2=0x7ffa18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa3364e640 1 --2- 172.21.15.116:0/3270731005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa180047c0 0x7ffa18004c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009ac0 tx=0x7ffa180057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa31beb640 1 -- 172.21.15.116:0/3270731005 reap_dead start 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa3364e640 1 -- 172.21.15.116:0/3270731005 shutdown_connections 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa3364e640 1 -- 172.21.15.116:0/3270731005 >> 172.21.15.116:0/3270731005 conn(0x7ffa2c0b5480 msgr2=0x7ffa2c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa3364e640 1 -- 172.21.15.116:0/3270731005 shutdown_connections 2024-02-18T11:48:29.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.159+0000 7ffa3364e640 1 -- 172.21.15.116:0/3270731005 wait complete. 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.160+0000 7ffa3364e640 1 Processor -- start 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.160+0000 7ffa3364e640 1 -- start start 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa3364e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa180047c0 0x7ffa2c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa3364e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0b82a0 0x7ffa2c15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa3364e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 0x7ffa2c0d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa3364e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0d42c0 con 0x7ffa2c0d3900 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa3364e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c15ecf0 con 0x7ffa180047c0 2024-02-18T11:48:29.163 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa3364e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c15f590 con 0x7ffa2c0b82a0 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 0x7ffa2c0d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 0x7ffa2c0d3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35020/0 (socket says 172.21.15.116:35020) 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 -- 172.21.15.116:0/1196304265 learned_addr learned my addr 172.21.15.116:0/1196304265 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa30be9640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0b82a0 0x7ffa2c15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa313ea640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa180047c0 0x7ffa2c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 -- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0b82a0 msgr2=0x7ffa2c15fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0b82a0 0x7ffa2c15fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 -- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa180047c0 msgr2=0x7ffa2c0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa180047c0 0x7ffa2c0c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 -- 172.21.15.116:0/1196304265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c15e150 con 0x7ffa2c0d3900 2024-02-18T11:48:29.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa31beb640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 0x7ffa2c0d3d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0cf4e0 tx=0x7ffa24004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.161+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24007c10 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.162+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2400f050 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.162+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c154ce0 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.162+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24014410 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.162+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c15f220 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.163+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c067840 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.163+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ffa24014570 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.163+0000 7ffa21ffb640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa1006bca0 0x7ffa1006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.163+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7ffa2408af10 con 0x7ffa2c0d3900 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.163+0000 7ffa313ea640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa1006bca0 0x7ffa1006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.164+0000 7ffa313ea640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa1006bca0 0x7ffa1006e160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005b50 tx=0x7ffa28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.168+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa240552d0 con 0x7ffa2c0d3900 2024-02-18T11:48:29.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.340+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa2c074520 con 0x7ffa2c0d3900 2024-02-18T11:48:29.333 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.341+0000 7ffa21ffb640 1 -- 172.21.15.116:0/1196304265 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa24058990 con 0x7ffa2c0d3900 2024-02-18T11:48:29.333 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa1006bca0 msgr2=0x7ffa1006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa1006bca0 0x7ffa1006e160 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa28005b50 tx=0x7ffa28009040 comp rx=0 tx=0).stop 2024-02-18T11:48:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 msgr2=0x7ffa2c0d3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 0x7ffa2c0d3d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0cf4e0 tx=0x7ffa24004700 comp rx=0 tx=0).stop 2024-02-18T11:48:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 shutdown_connections 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa1006bca0 0x7ffa1006e160 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa2c0d3900 0x7ffa2c0d3d80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa2c0b82a0 0x7ffa2c15fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 --2- 172.21.15.116:0/1196304265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa180047c0 0x7ffa2c0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 >> 172.21.15.116:0/1196304265 conn(0x7ffa2c0b5480 msgr2=0x7ffa2c0b7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 shutdown_connections 2024-02-18T11:48:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.343+0000 7ffa3364e640 1 -- 172.21.15.116:0/1196304265 wait complete. 2024-02-18T11:48:29.345 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:29.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.456+0000 7f005d519640 1 Processor -- start 2024-02-18T11:48:29.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.456+0000 7f005d519640 1 -- start start 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.457+0000 7f005d519640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 0x7f0058072d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.457+0000 7f005d519640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00580cd540 con 0x7f005806b270 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.457+0000 7f005d519640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00580b3de0 con 0x7f005815da30 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.457+0000 7f005d519640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005815de10 con 0x7f00580704a0 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.457+0000 7f00567fc640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f005806b270 0x7f0058165540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55584/0 (socket says 172.21.15.116:55584) 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.457+0000 7f00567fc640 1 -- 172.21.15.116:0/3139503661 learned_addr learned my addr 172.21.15.116:0/3139503661 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 0x7f0058072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309926388 0 0) 0x7f00580cd540 con 0x7f005806b270 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0040003520 con 0x7f005806b270 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1735720212 0 0) 0x7f0040003520 con 0x7f005806b270 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 msgr2=0x7f0058072d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 0x7f0058072d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 >> v1:172.21.15.143:6789/0 conn(0x7f005815da30 legacy=0x7f005806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00580d1470 con 0x7f005806b270 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f004c001670 con 0x7f005806b270 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 >> v1:172.21.15.116:6789/0 conn(0x7f005806b270 legacy=0x7f0058165540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 0x7f00400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040003ee0 0x7f00400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0040004d00 0x7f0040005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005815de10 con 0x7f0040004d00 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004000b9e0 con 0x7f0040003ee0 2024-02-18T11:48:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004000bba0 con 0x7f00580704a0 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0040004d00 0x7f0040005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f0056ffd640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 0x7f00400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00567fc640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040003ee0 0x7f00400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 -- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 msgr2=0x7f00400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00580704a0 0x7f00400039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 -- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040003ee0 msgr2=0x7f00400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0040003ee0 0x7f00400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.458+0000 7f00577fe640 1 -- 172.21.15.116:0/3139503661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040003520 con 0x7f0040004d00 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.459+0000 7f00577fe640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0040004d00 0x7f0040005180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0050009ac0 tx=0x7f00400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec0ac8abc00d8a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.459+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005002c6b0 con 0x7f0040004d00 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.459+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005002c810 con 0x7f0040004d00 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.459+0000 7f0055ffb640 1 -- 172.21.15.116:0/3139503661 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005002cba0 con 0x7f0040004d00 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f005d519640 1 -- 172.21.15.116:0/3139503661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0040004d00 msgr2=0x7f0040005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f005d519640 1 --2- 172.21.15.116:0/3139503661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0040004d00 0x7f0040005180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0050009ac0 tx=0x7f00400057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f00577fe640 1 -- 172.21.15.116:0/3139503661 reap_dead start 2024-02-18T11:48:29.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f005d519640 1 -- 172.21.15.116:0/3139503661 shutdown_connections 2024-02-18T11:48:29.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f005d519640 1 -- 172.21.15.116:0/3139503661 >> 172.21.15.116:0/3139503661 conn(0x7f00580b23f0 msgr2=0x7f00580b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f005d519640 1 -- 172.21.15.116:0/3139503661 shutdown_connections 2024-02-18T11:48:29.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.460+0000 7f005d519640 1 -- 172.21.15.116:0/3139503661 wait complete. 2024-02-18T11:48:29.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.461+0000 7f005d519640 1 Processor -- start 2024-02-18T11:48:29.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.461+0000 7f005d519640 1 -- start start 2024-02-18T11:48:29.457 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 0x7f005806e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f005815ebd0 0x7f005815f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00581560d0 0x7f0058156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00581f4960 con 0x7f005806e2a0 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058156ac0 con 0x7f005815ebd0 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00581f4680 con 0x7f00581560d0 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 0x7f005806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 0x7f005806e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35048/0 (socket says 172.21.15.116:35048) 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 -- 172.21.15.116:0/279280065 learned_addr learned my addr 172.21.15.116:0/279280065 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 -- 172.21.15.116:0/279280065 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00581560d0 msgr2=0x7f0058156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00581560d0 0x7f0058156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 -- 172.21.15.116:0/279280065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f005815ebd0 msgr2=0x7f005815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f005815ebd0 0x7f005815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 -- 172.21.15.116:0/279280065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005815e0f0 con 0x7f005806e2a0 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f0056ffd640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 0x7f005806e680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f004800fb60 tx=0x7f004800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048001b60 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004800dbb0 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005815e3d0 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.462+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048003970 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.463+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005806ebc0 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.464+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005815df50 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.464+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f0048003ad0 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.464+0000 7f003f7fe640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f004406bca0 0x7f004406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.464+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f004808af30 con 0x7f005806e2a0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.464+0000 7f00567fc640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f004406bca0 0x7f004406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.465+0000 7f00567fc640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f004406bca0 0x7f004406e160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f004c004840 tx=0x7f004c001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.461 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.469+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f005815df50 con 0x7f005806e2a0 2024-02-18T11:48:29.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.643+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f005815e8f0 con 0x7f005806e2a0 2024-02-18T11:48:29.636 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.644+0000 7f003f7fe640 1 -- 172.21.15.116:0/279280065 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f005815e8f0 con 0x7f005806e2a0 2024-02-18T11:48:29.636 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:29.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f004406bca0 msgr2=0x7f004406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f004406bca0 0x7f004406e160 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f004c004840 tx=0x7f004c001150 comp rx=0 tx=0).stop 2024-02-18T11:48:29.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 msgr2=0x7f005806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 0x7f005806e680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f004800fb60 tx=0x7f004800d520 comp rx=0 tx=0).stop 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 shutdown_connections 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f004406bca0 0x7f004406e160 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f00581560d0 0x7f0058156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f005815ebd0 0x7f005815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 --2- 172.21.15.116:0/279280065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f005806e2a0 0x7f005806e680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 >> 172.21.15.116:0/279280065 conn(0x7f00580b23f0 msgr2=0x7f00580b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 shutdown_connections 2024-02-18T11:48:29.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.646+0000 7f005d519640 1 -- 172.21.15.116:0/279280065 wait complete. 2024-02-18T11:48:29.649 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:29.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.761+0000 7f932e2ae640 1 Processor -- start 2024-02-18T11:48:29.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.761+0000 7f932e2ae640 1 -- start start 2024-02-18T11:48:29.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f932e2ae640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9328161fd0 0x7f932815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f932e2ae640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9328099a50 con 0x7f93280cd610 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f932e2ae640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9328099ca0 con 0x7f93280d0a80 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f932e2ae640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9328099f80 con 0x7f9328161fd0 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9327fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f93280cd610 0x7f93280d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55590/0 (socket says 172.21.15.116:55590) 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9327fff640 1 -- 172.21.15.116:0/970640967 learned_addr learned my addr 172.21.15.116:0/970640967 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031868851 0 0) 0x7f9328099a50 con 0x7f93280cd610 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9310003520 con 0x7f93280cd610 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091129647 0 0) 0x7f9310003520 con 0x7f93280cd610 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 >> v2:172.21.15.143:3300/0 conn(0x7f9328161fd0 msgr2=0x7f932815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 --2- 172.21.15.116:0/970640967 >> v2:172.21.15.143:3300/0 conn(0x7f9328161fd0 0x7f932815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 >> v1:172.21.15.143:6789/0 conn(0x7f93280d0a80 legacy=0x7f93281618c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932809a260 con 0x7f93280cd610 2024-02-18T11:48:29.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f9318001670 con 0x7f93280cd610 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 >> v1:172.21.15.116:6789/0 conn(0x7f93280cd610 legacy=0x7f93280d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310003970 0x7f9310003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100047c0 0x7f9310004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100052a0 0x7f931000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328099f80 con 0x7f93100052a0 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931000c190 con 0x7f9310003970 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931000c350 con 0x7f93100047c0 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f932c84b640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100052a0 0x7f931000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.762+0000 7f93277fe640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100047c0 0x7f9310004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 -- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100047c0 msgr2=0x7f9310004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100047c0 0x7f9310004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 -- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310003970 msgr2=0x7f9310003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310003970 0x7f9310003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 -- 172.21.15.116:0/970640967 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9310003520 con 0x7f93100052a0 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 -- 172.21.15.116:0/970640967 reap_dead start 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.763+0000 7f932c84b640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100052a0 0x7f931000b980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f93280c5200 tx=0x7f93100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebcb98a9583488e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932002cd10 con 0x7f93100052a0 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932002ce70 con 0x7f93100052a0 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f9326ffd640 1 -- 172.21.15.116:0/970640967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320003bb0 con 0x7f93100052a0 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 -- 172.21.15.116:0/970640967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100052a0 msgr2=0x7f931000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100052a0 0x7f931000b980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f93280c5200 tx=0x7f93100057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 -- 172.21.15.116:0/970640967 shutdown_connections 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 --2- 172.21.15.116:0/970640967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100052a0 0x7f931000b980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 -- 172.21.15.116:0/970640967 >> 172.21.15.116:0/970640967 conn(0x7f93280b31f0 msgr2=0x7f93280b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 -- 172.21.15.116:0/970640967 shutdown_connections 2024-02-18T11:48:29.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.764+0000 7f932e2ae640 1 -- 172.21.15.116:0/970640967 wait complete. 2024-02-18T11:48:29.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.765+0000 7f932e2ae640 1 Processor -- start 2024-02-18T11:48:29.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.765+0000 7f932e2ae640 1 -- start start 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932e2ae640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100052a0 0x7f932815ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932e2ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b64f0 0x7f93280b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932e2ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 0x7f93281608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932e2ae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328160e00 con 0x7f9328160440 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932e2ae640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b2070 con 0x7f93280b64f0 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932e2ae640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b2350 con 0x7f93100052a0 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932c84b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 0x7f93281608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932c84b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 0x7f93281608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35070/0 (socket says 172.21.15.116:35070) 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f932c84b640 1 -- 172.21.15.116:0/2254171480 learned_addr learned my addr 172.21.15.116:0/2254171480 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:29.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f9327fff640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100052a0 0x7f932815ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.766+0000 7f93277fe640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b64f0 0x7f93280b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932c84b640 1 -- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100052a0 msgr2=0x7f932815ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932c84b640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100052a0 0x7f932815ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932c84b640 1 -- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b64f0 msgr2=0x7f93280b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932c84b640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b64f0 0x7f93280b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932c84b640 1 -- 172.21.15.116:0/2254171480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b2630 con 0x7f9328160440 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f9327fff640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100052a0 0x7f932815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f93277fe640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b64f0 0x7f93280b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932c84b640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 0x7f93281608c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9320009ed0 tx=0x7f9320004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93200045f0 con 0x7f9328160440 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320004750 con 0x7f9328160440 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932003e660 con 0x7f9328160440 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280b2910 con 0x7f9328160440 2024-02-18T11:48:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.767+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932815f140 con 0x7f9328160440 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.768+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9328067840 con 0x7f9328160440 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.768+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f932003e7c0 con 0x7f9328160440 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.769+0000 7f9324ff9640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f931406bca0 0x7f931406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.769+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f93200ad5a0 con 0x7f9328160440 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.769+0000 7f9327fff640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f931406bca0 0x7f931406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.769+0000 7f9327fff640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f931406bca0 0x7f931406e160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9318000fb0 tx=0x7f9318001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.773+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9320077a80 con 0x7f9328160440 2024-02-18T11:48:29.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.942+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9328074520 con 0x7f9328160440 2024-02-18T11:48:29.935 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.943+0000 7f9324ff9640 1 -- 172.21.15.116:0/2254171480 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f932007b140 con 0x7f9328160440 2024-02-18T11:48:29.935 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.944+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f931406bca0 msgr2=0x7f931406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.944+0000 7f932e2ae640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f931406bca0 0x7f931406e160 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9318000fb0 tx=0x7f9318001700 comp rx=0 tx=0).stop 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.944+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 msgr2=0x7f93281608c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.944+0000 7f932e2ae640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 0x7f93281608c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9320009ed0 tx=0x7f9320004b50 comp rx=0 tx=0).stop 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 shutdown_connections 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f931406bca0 0x7f931406e160 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328160440 0x7f93281608c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93280b64f0 0x7f93280b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 --2- 172.21.15.116:0/2254171480 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f93100052a0 0x7f932815ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 >> 172.21.15.116:0/2254171480 conn(0x7f93280b31f0 msgr2=0x7f93280b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:29.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 shutdown_connections 2024-02-18T11:48:29.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:29.945+0000 7f932e2ae640 1 -- 172.21.15.116:0/2254171480 wait complete. 2024-02-18T11:48:29.948 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:48:30.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.056+0000 7f861d948640 1 Processor -- start 2024-02-18T11:48:30.048 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.056+0000 7f861d948640 1 -- start start 2024-02-18T11:48:30.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f861d948640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f861815dbf0 0x7f86181617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f861d948640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86180d1ee0 con 0x7f86180b9370 2024-02-18T11:48:30.051 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f861d948640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8618161dc0 con 0x7f86180cee50 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f861d948640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86181620a0 con 0x7f861815dbf0 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f86167fc640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f86180b9370 0x7f86180ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55596/0 (socket says 172.21.15.116:55596) 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f86167fc640 1 -- 172.21.15.116:0/1903750056 learned_addr learned my addr 172.21.15.116:0/1903750056 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8616ffd640 1 --2- 172.21.15.116:0/1903750056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f861815dbf0 0x7f86181617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8615ffb640 1 -- 172.21.15.116:0/1903750056 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044311715 0 0) 0x7f86180d1ee0 con 0x7f86180b9370 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8615ffb640 1 -- 172.21.15.116:0/1903750056 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8604003520 con 0x7f86180b9370 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8615ffb640 1 -- 172.21.15.116:0/1903750056 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 373327800 0 0) 0x7f8604003520 con 0x7f86180b9370 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8616ffd640 1 -- 172.21.15.116:0/1903750056 >> v1:172.21.15.143:6789/0 conn(0x7f86180cee50 legacy=0x7f86180d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8616ffd640 1 -- 172.21.15.116:0/1903750056 >> v1:172.21.15.116:6789/0 conn(0x7f86180b9370 legacy=0x7f86180ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.057+0000 7f8616ffd640 1 -- 172.21.15.116:0/1903750056 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618162380 con 0x7f861815dbf0 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.058+0000 7f8616ffd640 1 --2- 172.21.15.116:0/1903750056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f861815dbf0 0x7f86181617e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8608009dc0 tx=0x7f860802bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6508e134c20d8a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.052 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.058+0000 7f8615ffb640 1 -- 172.21.15.116:0/1903750056 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600001670 con 0x7f861815dbf0 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.058+0000 7f8615ffb640 1 -- 172.21.15.116:0/1903750056 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860802c770 con 0x7f861815dbf0 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.058+0000 7f8615ffb640 1 -- 172.21.15.116:0/1903750056 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860802cb00 con 0x7f861815dbf0 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 -- 172.21.15.116:0/1903750056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f861815dbf0 msgr2=0x7f86181617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 --2- 172.21.15.116:0/1903750056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f861815dbf0 0x7f86181617e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8608009dc0 tx=0x7f860802bc40 comp rx=0 tx=0).stop 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 -- 172.21.15.116:0/1903750056 shutdown_connections 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 --2- 172.21.15.116:0/1903750056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f861815dbf0 0x7f86181617e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 -- 172.21.15.116:0/1903750056 >> 172.21.15.116:0/1903750056 conn(0x7f86180b5480 msgr2=0x7f86180b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 -- 172.21.15.116:0/1903750056 shutdown_connections 2024-02-18T11:48:30.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.059+0000 7f861d948640 1 -- 172.21.15.116:0/1903750056 wait complete. 2024-02-18T11:48:30.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.060+0000 7f861d948640 1 Processor -- start 2024-02-18T11:48:30.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.060+0000 7f861d948640 1 -- start start 2024-02-18T11:48:30.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f861d948640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 0x7f86181522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f861d948640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86180cee50 0x7f86181f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f861d948640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180ab000 0x7f86180ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f861d948640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861815bdb0 con 0x7f86180b9370 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f861d948640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86180ab9c0 con 0x7f86180ab000 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f861d948640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8618159a70 con 0x7f86180cee50 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 0x7f86181522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 0x7f86181522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35080/0 (socket says 172.21.15.116:35080) 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 -- 172.21.15.116:0/233244034 learned_addr learned my addr 172.21.15.116:0/233244034 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f86167fc640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86180cee50 0x7f86181f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f86177fe640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180ab000 0x7f86180ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 -- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86180cee50 msgr2=0x7f86181f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86180cee50 0x7f86181f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 -- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180ab000 msgr2=0x7f86180ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180ab000 0x7f86180ab480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f8616ffd640 1 -- 172.21.15.116:0/233244034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608009a70 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.061+0000 7f86177fe640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180ab000 0x7f86180ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.062+0000 7f8616ffd640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 0x7f86181522b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8608037040 tx=0x7f8608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.062+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608034c80 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.062+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8608034e20 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.062+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8608003bd0 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.062+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861815e150 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.062+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861815a030 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.063+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8618067840 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.063+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f860803b050 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.063+0000 7f860ffff640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f85f806bc70 0x7f85f806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.064+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f86080acd60 con 0x7f86180b9370 2024-02-18T11:48:30.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.064+0000 7f86167fc640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f85f806bc70 0x7f85f806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.064+0000 7f86167fc640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f85f806bc70 0x7f85f806e130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8600001eb0 tx=0x7f8600001c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.068+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8608077120 con 0x7f86180b9370 2024-02-18T11:48:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.249+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f86180b9790 con 0x7f86180b9370 2024-02-18T11:48:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.250+0000 7f860ffff640 1 -- 172.21.15.116:0/233244034 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f860807a7e0 con 0x7f86180b9370 2024-02-18T11:48:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T11:48:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f85f806bc70 msgr2=0x7f85f806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f85f806bc70 0x7f85f806e130 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8600001eb0 tx=0x7f8600001c80 comp rx=0 tx=0).stop 2024-02-18T11:48:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 msgr2=0x7f86181522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 0x7f86181522b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8608037040 tx=0x7f8608004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 shutdown_connections 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f85f806bc70 0x7f85f806e130 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86180ab000 0x7f86180ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f86180cee50 0x7f86181f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 --2- 172.21.15.116:0/233244034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86180b9370 0x7f86181522b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 >> 172.21.15.116:0/233244034 conn(0x7f86180b5480 msgr2=0x7f86180cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 shutdown_connections 2024-02-18T11:48:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.252+0000 7f861d948640 1 -- 172.21.15.116:0/233244034 wait complete. 2024-02-18T11:48:30.255 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-02-18T11:48:30.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:30.255 DEBUG:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-02-18T11:48:30.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T11:48:30.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:30.283 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-02-18T11:48:30.338 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:30.338 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:30.358 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:30.358 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:30.416 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T11:48:30.416 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:30.474 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:30.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.585+0000 7fd5fdbb5640 1 Processor -- start 2024-02-18T11:48:30.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.585+0000 7fd5fdbb5640 1 -- start start 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5fdbb5640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5f815ddc0 0x7fd5f8161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5fdbb5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f80b6b00 0x7fd5f8161d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5fdbb5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5f80b57d0 0x7fd5f80b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5fdbb5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80b6170 con 0x7fd5f80b57d0 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5fdbb5640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f815e1a0 con 0x7fd5f815ddc0 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5fdbb5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80b96a0 con 0x7fd5f80b6b00 2024-02-18T11:48:30.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5f80b57d0 0x7fd5f80b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5f80b57d0 0x7fd5f80b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35094/0 (socket says 172.21.15.116:35094) 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 -- 172.21.15.116:0/2409393467 learned_addr learned my addr 172.21.15.116:0/2409393467 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f6ffd640 1 --2- 172.21.15.116:0/2409393467 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f80b6b00 0x7fd5f8161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f77fe640 1 --2- 172.21.15.116:0/2409393467 >> v2:172.21.15.143:3300/0 conn(0x7fd5f815ddc0 0x7fd5f8161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 -- 172.21.15.116:0/2409393467 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f80b6b00 msgr2=0x7fd5f8161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 --2- 172.21.15.116:0/2409393467 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f80b6b00 0x7fd5f8161d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 -- 172.21.15.116:0/2409393467 >> v2:172.21.15.143:3300/0 conn(0x7fd5f815ddc0 msgr2=0x7fd5f8161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 --2- 172.21.15.116:0/2409393467 >> v2:172.21.15.143:3300/0 conn(0x7fd5f815ddc0 0x7fd5f8161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.586+0000 7fd5f7fff640 1 -- 172.21.15.116:0/2409393467 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f81623e0 con 0x7fd5f80b57d0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.587+0000 7fd5f7fff640 1 --2- 172.21.15.116:0/2409393467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b57d0 0x7fd5f80b5c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8073270 tx=0x7fd5f002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a309b6311934f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.587+0000 7fd5f67fc640 1 -- 172.21.15.116:0/2409393467 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f002ca70 con 0x7fd5f80b57d0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.587+0000 7fd5f67fc640 1 -- 172.21.15.116:0/2409393467 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f002cbd0 con 0x7fd5f80b57d0 2024-02-18T11:48:30.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.587+0000 7fd5f67fc640 1 -- 172.21.15.116:0/2409393467 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f00343f0 con 0x7fd5f80b57d0 2024-02-18T11:48:30.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/2409393467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b57d0 msgr2=0x7fd5f80b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/2409393467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b57d0 0x7fd5f80b5c30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8073270 tx=0x7fd5f002bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/2409393467 shutdown_connections 2024-02-18T11:48:30.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/2409393467 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f80b6b00 0x7fd5f8161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.583 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/2409393467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b57d0 0x7fd5f80b5c30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/2409393467 >> v2:172.21.15.143:3300/0 conn(0x7fd5f815ddc0 0x7fd5f8161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/2409393467 >> 172.21.15.116:0/2409393467 conn(0x7fd5f80d0870 msgr2=0x7fd5f80d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/2409393467 shutdown_connections 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.588+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/2409393467 wait complete. 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.589+0000 7fd5fdbb5640 1 Processor -- start 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.589+0000 7fd5fdbb5640 1 -- start start 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5f80b57d0 0x7fd5f81558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 0x7fd5f81f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f815ddc0 0x7fd5f8157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8151d70 con 0x7fd5f80b6b00 2024-02-18T11:48:30.584 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8156c60 con 0x7fd5f80b57d0 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8156f40 con 0x7fd5f815ddc0 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 0x7fd5f81f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 0x7fd5f81f2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35098/0 (socket says 172.21.15.116:35098) 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 -- 172.21.15.116:0/3711372961 learned_addr learned my addr 172.21.15.116:0/3711372961 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f7fff640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f815ddc0 0x7fd5f8157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f77fe640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5f80b57d0 0x7fd5f81558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 -- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f815ddc0 msgr2=0x7fd5f8157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f815ddc0 0x7fd5f8157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 -- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5f80b57d0 msgr2=0x7fd5f81558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5f80b57d0 0x7fd5f81558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 -- 172.21.15.116:0/3711372961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0009b30 con 0x7fd5f80b6b00 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f7fff640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f815ddc0 0x7fd5f8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f77fe640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5f80b57d0 0x7fd5f81558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:30.585 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5f6ffd640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 0x7fd5f81f2c40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d7d0 tx=0x7fd5ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0096a0 con 0x7fd5f80b6b00 2024-02-18T11:48:30.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec009800 con 0x7fd5f80b6b00 2024-02-18T11:48:30.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f8157220 con 0x7fd5f80b6b00 2024-02-18T11:48:30.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec009b90 con 0x7fd5f80b6b00 2024-02-18T11:48:30.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.590+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f81568e0 con 0x7fd5f80b6b00 2024-02-18T11:48:30.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.591+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f80d16e0 con 0x7fd5f80b6b00 2024-02-18T11:48:30.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.592+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd5ec009cf0 con 0x7fd5f80b6b00 2024-02-18T11:48:30.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.592+0000 7fd5dffff640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd5e406bc70 0x7fd5e406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.592+0000 7fd5f77fe640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd5e406bc70 0x7fd5e406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.592+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd5ec08b010 con 0x7fd5f80b6b00 2024-02-18T11:48:30.587 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.593+0000 7fd5f77fe640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd5e406bc70 0x7fd5e406e130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006fd0 tx=0x7fd5e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.597+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5ec055460 con 0x7fd5f80b6b00 2024-02-18T11:48:30.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.771+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5f8161ee0 con 0x7fd5f80b6b00 2024-02-18T11:48:30.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.772+0000 7fd5dffff640 1 -- 172.21.15.116:0/3711372961 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7fd5ec058b20 con 0x7fd5f80b6b00 2024-02-18T11:48:30.765 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:30.765 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd5e406bc70 msgr2=0x7fd5e406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd5e406bc70 0x7fd5e406e130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006fd0 tx=0x7fd5e8008040 comp rx=0 tx=0).stop 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 msgr2=0x7fd5f81f2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 0x7fd5f81f2c40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00d7d0 tx=0x7fd5ec00dc90 comp rx=0 tx=0).stop 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 shutdown_connections 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd5e406bc70 0x7fd5e406e130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5f815ddc0 0x7fd5f8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5f80b6b00 0x7fd5f81f2c40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 --2- 172.21.15.116:0/3711372961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5f80b57d0 0x7fd5f81558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 >> 172.21.15.116:0/3711372961 conn(0x7fd5f80d0870 msgr2=0x7fd5f80b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:30.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 shutdown_connections 2024-02-18T11:48:30.768 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.774+0000 7fd5fdbb5640 1 -- 172.21.15.116:0/3711372961 wait complete. 2024-02-18T11:48:30.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:30.778 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:30.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.889+0000 7fe8e6664640 1 Processor -- start 2024-02-18T11:48:30.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.889+0000 7fe8e6664640 1 -- start start 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8e6664640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8e0145230 0x7fe8e015be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8e6664640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8e015c3d0 0x7fe8e00711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8e6664640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e0071b30 con 0x7fe8e0071700 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8e6664640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e015d050 con 0x7fe8e0145230 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8e6664640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e0145610 con 0x7fe8e015c3d0 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8e0145230 0x7fe8e015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.883 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe8e0145230 0x7fe8e015be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35106/0 (socket says 172.21.15.116:35106) 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 -- 172.21.15.116:0/484749828 learned_addr learned my addr 172.21.15.116:0/484749828 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8df7fe640 1 --2- 172.21.15.116:0/484749828 >> v2:172.21.15.143:3300/0 conn(0x7fe8e015c3d0 0x7fe8e00711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 -- 172.21.15.116:0/484749828 >> v2:172.21.15.143:3300/0 conn(0x7fe8e015c3d0 msgr2=0x7fe8e00711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 --2- 172.21.15.116:0/484749828 >> v2:172.21.15.143:3300/0 conn(0x7fe8e015c3d0 0x7fe8e00711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 -- 172.21.15.116:0/484749828 >> v1:172.21.15.143:6789/0 conn(0x7fe8e0071700 legacy=0x7fe8e006d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.890+0000 7fe8dffff640 1 -- 172.21.15.116:0/484749828 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e015c960 con 0x7fe8e0145230 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.891+0000 7fe8dffff640 1 --2- 172.21.15.116:0/484749828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0145230 0x7fe8e015be90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00b4990 tx=0x7fe8d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aff3034c3702033 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.891+0000 7fe8deffd640 1 -- 172.21.15.116:0/484749828 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4031640 con 0x7fe8e0145230 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.891+0000 7fe8deffd640 1 -- 172.21.15.116:0/484749828 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d40317a0 con 0x7fe8e0145230 2024-02-18T11:48:30.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.891+0000 7fe8deffd640 1 -- 172.21.15.116:0/484749828 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4031b30 con 0x7fe8e0145230 2024-02-18T11:48:30.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 -- 172.21.15.116:0/484749828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0145230 msgr2=0x7fe8e015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 --2- 172.21.15.116:0/484749828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0145230 0x7fe8e015be90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00b4990 tx=0x7fe8d402b8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 -- 172.21.15.116:0/484749828 shutdown_connections 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 --2- 172.21.15.116:0/484749828 >> v2:172.21.15.143:3300/0 conn(0x7fe8e015c3d0 0x7fe8e00711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 --2- 172.21.15.116:0/484749828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0145230 0x7fe8e015be90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 -- 172.21.15.116:0/484749828 >> 172.21.15.116:0/484749828 conn(0x7fe8e0067000 msgr2=0x7fe8e006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 -- 172.21.15.116:0/484749828 shutdown_connections 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.892+0000 7fe8e6664640 1 -- 172.21.15.116:0/484749828 wait complete. 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.893+0000 7fe8e6664640 1 Processor -- start 2024-02-18T11:48:30.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.893+0000 7fe8e6664640 1 -- start start 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 0x7fe8e015b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8e0145230 0x7fe8e00adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8e015c3d0 0x7fe8e00ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c5450 con 0x7fe8e0071700 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c5730 con 0x7fe8e0145230 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c5a10 con 0x7fe8e015c3d0 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 0x7fe8e015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 0x7fe8e015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35120/0 (socket says 172.21.15.116:35120) 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 -- 172.21.15.116:0/76768271 learned_addr learned my addr 172.21.15.116:0/76768271 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 -- 172.21.15.116:0/76768271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8e015c3d0 msgr2=0x7fe8e00ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8e015c3d0 0x7fe8e00ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 -- 172.21.15.116:0/76768271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8e0145230 msgr2=0x7fe8e00adbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8e0145230 0x7fe8e00adbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 -- 172.21.15.116:0/76768271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40094d0 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dffff640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 0x7fe8e015b210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000c00 tx=0x7fe8d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d403ac60 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d403adc0 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4044610 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e00c5cf0 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.894+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e015b750 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.895+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e00c6210 con 0x7fe8e0071700 2024-02-18T11:48:30.889 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.896+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe8d402f050 con 0x7fe8e0071700 2024-02-18T11:48:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.896+0000 7fe8dcff9640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe8cc06bc70 0x7fe8cc06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.896+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe8d40acaf0 con 0x7fe8e0071700 2024-02-18T11:48:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.896+0000 7fe8df7fe640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe8cc06bc70 0x7fe8cc06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:30.890 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.897+0000 7fe8df7fe640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe8cc06bc70 0x7fe8cc06e130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0006fd0 tx=0x7fe8d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:30.892 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:30.901+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8e00c6210 con 0x7fe8e0071700 2024-02-18T11:48:31.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.073+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8e00c6210 con 0x7fe8e0071700 2024-02-18T11:48:31.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.073+0000 7fe8dcff9640 1 -- 172.21.15.116:0/76768271 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8e00c6210 con 0x7fe8e0071700 2024-02-18T11:48:31.066 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe8cc06bc70 msgr2=0x7fe8cc06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe8cc06bc70 0x7fe8cc06e130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0006fd0 tx=0x7fe8d0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 msgr2=0x7fe8e015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 0x7fe8e015b210 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4000c00 tx=0x7fe8d40046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 shutdown_connections 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe8cc06bc70 0x7fe8cc06e130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8e015c3d0 0x7fe8e00ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8e0145230 0x7fe8e00adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 --2- 172.21.15.116:0/76768271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8e0071700 0x7fe8e015b210 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 >> 172.21.15.116:0/76768271 conn(0x7fe8e0067000 msgr2=0x7fe8e015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 shutdown_connections 2024-02-18T11:48:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.075+0000 7fe8e6664640 1 -- 172.21.15.116:0/76768271 wait complete. 2024-02-18T11:48:31.078 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:31.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.189+0000 7f33a0913640 1 Processor -- start 2024-02-18T11:48:31.181 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.189+0000 7f33a0913640 1 -- start start 2024-02-18T11:48:31.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f33a0913640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f339c0b31a0 0x7f339c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f33a0913640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c0b3610 con 0x7f339c144090 2024-02-18T11:48:31.182 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f33a0913640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c06eca0 con 0x7f339c06e870 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f33a0913640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339c0b2a50 con 0x7f339c0b31a0 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f339a59c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f339c144090 0x7f339c0cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55608/0 (socket says 172.21.15.116:55608) 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f339a59c640 1 -- 172.21.15.116:0/2867232766 learned_addr learned my addr 172.21.15.116:0/2867232766 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f3399d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f339c0b31a0 0x7f339c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f3399d9b640 1 -- 172.21.15.116:0/2867232766 >> v1:172.21.15.143:6789/0 conn(0x7f339c06e870 legacy=0x7f339c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f3399d9b640 1 -- 172.21.15.116:0/2867232766 >> v1:172.21.15.116:6789/0 conn(0x7f339c144090 legacy=0x7f339c0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.190+0000 7f3399d9b640 1 -- 172.21.15.116:0/2867232766 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0cd0f0 con 0x7f339c0b31a0 2024-02-18T11:48:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.191+0000 7f3399d9b640 1 --2- 172.21.15.116:0/2867232766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c06e330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33900097a0 tx=0x7f339002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3403a144e8654 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.191+0000 7f339959a640 1 -- 172.21.15.116:0/2867232766 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339002e070 con 0x7f339c0b31a0 2024-02-18T11:48:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.191+0000 7f339959a640 1 -- 172.21.15.116:0/2867232766 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33900315a0 con 0x7f339c0b31a0 2024-02-18T11:48:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.191+0000 7f339959a640 1 -- 172.21.15.116:0/2867232766 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390031930 con 0x7f339c0b31a0 2024-02-18T11:48:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 -- 172.21.15.116:0/2867232766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 msgr2=0x7f339c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 --2- 172.21.15.116:0/2867232766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c06e330 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33900097a0 tx=0x7f339002b840 comp rx=0 tx=0).stop 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 -- 172.21.15.116:0/2867232766 shutdown_connections 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 --2- 172.21.15.116:0/2867232766 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c06e330 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 -- 172.21.15.116:0/2867232766 >> 172.21.15.116:0/2867232766 conn(0x7f339c067000 msgr2=0x7f339c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 -- 172.21.15.116:0/2867232766 shutdown_connections 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.192+0000 7f33a0913640 1 -- 172.21.15.116:0/2867232766 wait complete. 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.193+0000 7f33a0913640 1 Processor -- start 2024-02-18T11:48:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.193+0000 7f33a0913640 1 -- start start 2024-02-18T11:48:31.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f33a0913640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f339c06e870 0x7f339c159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f33a0913640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c0af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f33a0913640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f339c144090 0x7f339c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f33a0913640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c1ef590 con 0x7f339c0b31a0 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f33a0913640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c15a1a0 con 0x7f339c144090 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f33a0913640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c132530 con 0x7f339c06e870 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c0af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c0af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35148/0 (socket says 172.21.15.116:35148) 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 -- 172.21.15.116:0/822726329 learned_addr learned my addr 172.21.15.116:0/822726329 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 -- 172.21.15.116:0/822726329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f339c06e870 msgr2=0x7f339c159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f339c06e870 0x7f339c159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 -- 172.21.15.116:0/822726329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f339c144090 msgr2=0x7f339c0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f339c144090 0x7f339c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 -- 172.21.15.116:0/822726329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009400 con 0x7f339c0b31a0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.194+0000 7f3399d9b640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c0af260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3390034040 tx=0x7f33900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.195+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390032900 con 0x7f339c0b31a0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.195+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3390032a80 con 0x7f339c0b31a0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.195+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c132810 con 0x7f339c0b31a0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.195+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390032e30 con 0x7f339c0b31a0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.195+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c132dd0 con 0x7f339c0b31a0 2024-02-18T11:48:31.189 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.196+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c06dfd0 con 0x7f339c0b31a0 2024-02-18T11:48:31.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.196+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f339003b3f0 con 0x7f339c0b31a0 2024-02-18T11:48:31.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.196+0000 7f338affd640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f337c06bc70 0x7f337c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.197+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f33900acc70 con 0x7f339c0b31a0 2024-02-18T11:48:31.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.197+0000 7f339a59c640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f337c06bc70 0x7f337c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.190 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.197+0000 7f339a59c640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f337c06bc70 0x7f337c06e130 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f338c0012e0 tx=0x7f338c001590 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.193 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.201+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3390077030 con 0x7f339c0b31a0 2024-02-18T11:48:31.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.370+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f339c074520 con 0x7f339c0b31a0 2024-02-18T11:48:31.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.371+0000 7f338affd640 1 -- 172.21.15.116:0/822726329 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f339007a6f0 con 0x7f339c0b31a0 2024-02-18T11:48:31.363 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.372+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f337c06bc70 msgr2=0x7f337c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.372+0000 7f33a0913640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f337c06bc70 0x7f337c06e130 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f338c0012e0 tx=0x7f338c001590 comp rx=0 tx=0).stop 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.372+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 msgr2=0x7f339c0af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.372+0000 7f33a0913640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c0af260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3390034040 tx=0x7f33900046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 shutdown_connections 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f337c06bc70 0x7f337c06e130 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f339c144090 0x7f339c0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f339c0b31a0 0x7f339c0af260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 --2- 172.21.15.116:0/822726329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f339c06e870 0x7f339c159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 >> 172.21.15.116:0/822726329 conn(0x7f339c067000 msgr2=0x7f339c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 shutdown_connections 2024-02-18T11:48:31.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.373+0000 7f33a0913640 1 -- 172.21.15.116:0/822726329 wait complete. 2024-02-18T11:48:31.376 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:31.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.486+0000 7f57dc40e640 1 Processor -- start 2024-02-18T11:48:31.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.486+0000 7f57dc40e640 1 -- start start 2024-02-18T11:48:31.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.487+0000 7f57dc40e640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 0x7f57d4070be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.487+0000 7f57dc40e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 0x7f57d4071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.487+0000 7f57dc40e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40ceed0 con 0x7f57d40719d0 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.487+0000 7f57dc40e640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d40b1100 con 0x7f57d4145250 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.487+0000 7f57dc40e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40ce2c0 con 0x7f57d40b5690 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da9ab640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 0x7f57d4071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da1aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 0x7f57d4070be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57d99a9640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f57d40719d0 0x7f57d415f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:56144/0 (socket says 172.21.15.116:56144) 2024-02-18T11:48:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da1aa640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 0x7f57d4070be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:52896/0 (socket says 172.21.15.116:52896) 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da9ab640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 0x7f57d4071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:39394/0 (socket says 172.21.15.116:39394) 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57d99a9640 1 -- 172.21.15.116:0/974248329 learned_addr learned my addr 172.21.15.116:0/974248329 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da9ab640 1 -- 172.21.15.116:0/974248329 >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 msgr2=0x7f57d4070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da9ab640 1 --2- 172.21.15.116:0/974248329 >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 0x7f57d4070be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da9ab640 1 -- 172.21.15.116:0/974248329 >> v1:172.21.15.143:6789/0 conn(0x7f57d40719d0 legacy=0x7f57d415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da9ab640 1 -- 172.21.15.116:0/974248329 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40ce5a0 con 0x7f57d40b5690 2024-02-18T11:48:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.488+0000 7f57da1aa640 1 --2- 172.21.15.116:0/974248329 >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 0x7f57d4070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.489+0000 7f57da9ab640 1 --2- 172.21.15.116:0/974248329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 0x7f57d4071120 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009d90 tx=0x7f57cc02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd798115f265b8d server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.489+0000 7f57d91a8640 1 -- 172.21.15.116:0/974248329 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc02cbc0 con 0x7f57d40b5690 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.489+0000 7f57d91a8640 1 -- 172.21.15.116:0/974248329 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc02cd20 con 0x7f57d40b5690 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.489+0000 7f57d91a8640 1 -- 172.21.15.116:0/974248329 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc034660 con 0x7f57d40b5690 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 -- 172.21.15.116:0/974248329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 msgr2=0x7f57d4071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 --2- 172.21.15.116:0/974248329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 0x7f57d4071120 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009d90 tx=0x7f57cc02be10 comp rx=0 tx=0).stop 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 -- 172.21.15.116:0/974248329 shutdown_connections 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 --2- 172.21.15.116:0/974248329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d40b5690 0x7f57d4071120 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 --2- 172.21.15.116:0/974248329 >> v2:172.21.15.143:3300/0 conn(0x7f57d4145250 0x7f57d4070be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 -- 172.21.15.116:0/974248329 >> 172.21.15.116:0/974248329 conn(0x7f57d406f460 msgr2=0x7f57d406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 -- 172.21.15.116:0/974248329 shutdown_connections 2024-02-18T11:48:31.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.490+0000 7f57dc40e640 1 -- 172.21.15.116:0/974248329 wait complete. 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.491+0000 7f57dc40e640 1 Processor -- start 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.491+0000 7f57dc40e640 1 -- start start 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 0x7f57d415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57d40b5690 0x7f57d40d1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d4145250 0x7f57d40d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41e4120 con 0x7f57d40719d0 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41e43c0 con 0x7f57d40b5690 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41e46a0 con 0x7f57d4145250 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 0x7f57d415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 0x7f57d415ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35152/0 (socket says 172.21.15.116:35152) 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 -- 172.21.15.116:0/2047718481 learned_addr learned my addr 172.21.15.116:0/2047718481 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57d99a9640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57d40b5690 0x7f57d40d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da9ab640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d4145250 0x7f57d40d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 -- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d4145250 msgr2=0x7f57d40d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d4145250 0x7f57d40d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 -- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57d40b5690 msgr2=0x7f57d40d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57d40b5690 0x7f57d40d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 -- 172.21.15.116:0/2047718481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0009400 con 0x7f57d40719d0 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57da1aa640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 0x7f57d415ab70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e960 tx=0x7f57d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d000ca00 con 0x7f57d40719d0 2024-02-18T11:48:31.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d000cb60 con 0x7f57d40719d0 2024-02-18T11:48:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0013470 con 0x7f57d40719d0 2024-02-18T11:48:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc009a70 con 0x7f57d40719d0 2024-02-18T11:48:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.492+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d41e4c60 con 0x7f57d40719d0 2024-02-18T11:48:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.493+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d40cf7a0 con 0x7f57d40719d0 2024-02-18T11:48:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.494+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f57d00135d0 con 0x7f57d40719d0 2024-02-18T11:48:31.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.494+0000 7f57caffd640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57b806bc70 0x7f57b806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.494+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f57d008aeb0 con 0x7f57d40719d0 2024-02-18T11:48:31.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.494+0000 7f57d99a9640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57b806bc70 0x7f57b806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.495+0000 7f57d99a9640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57b806bc70 0x7f57b806e130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f57d41f4a70 tx=0x7f57c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.499+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57d00551e0 con 0x7f57d40719d0 2024-02-18T11:48:31.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.669+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57d41e4df0 con 0x7f57d40719d0 2024-02-18T11:48:31.661 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.669+0000 7f57caffd640 1 -- 172.21.15.116:0/2047718481 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57d00210a0 con 0x7f57d40719d0 2024-02-18T11:48:31.662 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:31.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57b806bc70 msgr2=0x7f57b806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57b806bc70 0x7f57b806e130 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f57d41f4a70 tx=0x7f57c4009040 comp rx=0 tx=0).stop 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 msgr2=0x7f57d415ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 0x7f57d415ab70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f57d000e960 tx=0x7f57d000ee20 comp rx=0 tx=0).stop 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 shutdown_connections 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f57b806bc70 0x7f57b806e130 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f57d4145250 0x7f57d40d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57d40b5690 0x7f57d40d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 --2- 172.21.15.116:0/2047718481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d40719d0 0x7f57d415ab70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 >> 172.21.15.116:0/2047718481 conn(0x7f57d406f460 msgr2=0x7f57d40b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 shutdown_connections 2024-02-18T11:48:31.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.671+0000 7f57dc40e640 1 -- 172.21.15.116:0/2047718481 wait complete. 2024-02-18T11:48:31.674 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:31.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.786+0000 7fe295b48640 1 Processor -- start 2024-02-18T11:48:31.778 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.786+0000 7fe295b48640 1 -- start start 2024-02-18T11:48:31.779 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe295b48640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900af8a0 0x7fe2900b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe295b48640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe2900afc80 0x7fe2900cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe295b48640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900c6cc0 con 0x7fe29006b700 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe295b48640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900cce80 con 0x7fe2900afc80 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe295b48640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c6fa0 con 0x7fe2900af8a0 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28ffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe29006b700 0x7fe29006ab60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55620/0 (socket says 172.21.15.116:55620) 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28ffff640 1 -- 172.21.15.116:0/4010208741 learned_addr learned my addr 172.21.15.116:0/4010208741 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28f7fe640 1 --2- 172.21.15.116:0/4010208741 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900af8a0 0x7fe2900b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28effd640 1 --2- 172.21.15.116:0/4010208741 >> v2:172.21.15.143:3300/0 conn(0x7fe2900afc80 0x7fe2900cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28e7fc640 1 -- 172.21.15.116:0/4010208741 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519935122 0 0) 0x7fe2900c6cc0 con 0x7fe29006b700 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28e7fc640 1 -- 172.21.15.116:0/4010208741 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe278003520 con 0x7fe29006b700 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28f7fe640 1 -- 172.21.15.116:0/4010208741 >> v2:172.21.15.143:3300/0 conn(0x7fe2900afc80 msgr2=0x7fe2900cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:31.780 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28f7fe640 1 --2- 172.21.15.116:0/4010208741 >> v2:172.21.15.143:3300/0 conn(0x7fe2900afc80 0x7fe2900cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28f7fe640 1 -- 172.21.15.116:0/4010208741 >> v1:172.21.15.116:6789/0 conn(0x7fe29006b700 legacy=0x7fe29006ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.781 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.787+0000 7fe28f7fe640 1 -- 172.21.15.116:0/4010208741 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900c7280 con 0x7fe2900af8a0 2024-02-18T11:48:31.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.788+0000 7fe28f7fe640 1 --2- 172.21.15.116:0/4010208741 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900af8a0 0x7fe2900b2d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097a0 tx=0x7fe28002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4738f570bc45e8f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.788+0000 7fe28e7fc640 1 -- 172.21.15.116:0/4010208741 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280031830 con 0x7fe2900af8a0 2024-02-18T11:48:31.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.788+0000 7fe28e7fc640 1 -- 172.21.15.116:0/4010208741 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe280031990 con 0x7fe2900af8a0 2024-02-18T11:48:31.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.788+0000 7fe28e7fc640 1 -- 172.21.15.116:0/4010208741 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280031d20 con 0x7fe2900af8a0 2024-02-18T11:48:31.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 -- 172.21.15.116:0/4010208741 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900af8a0 msgr2=0x7fe2900b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.782 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 --2- 172.21.15.116:0/4010208741 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900af8a0 0x7fe2900b2d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097a0 tx=0x7fe28002b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 -- 172.21.15.116:0/4010208741 shutdown_connections 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 --2- 172.21.15.116:0/4010208741 >> v2:172.21.15.143:3300/0 conn(0x7fe2900afc80 0x7fe2900cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 --2- 172.21.15.116:0/4010208741 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900af8a0 0x7fe2900b2d50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 -- 172.21.15.116:0/4010208741 >> 172.21.15.116:0/4010208741 conn(0x7fe2900b2230 msgr2=0x7fe2900b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 -- 172.21.15.116:0/4010208741 shutdown_connections 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.789+0000 7fe295b48640 1 -- 172.21.15.116:0/4010208741 wait complete. 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.790+0000 7fe295b48640 1 Processor -- start 2024-02-18T11:48:31.783 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.790+0000 7fe295b48640 1 -- start start 2024-02-18T11:48:31.785 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe295b48640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe29006b700 0x7fe2900b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe295b48640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 0x7fe29014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe295b48640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900afc80 0x7fe2900b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe295b48640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901f7210 con 0x7fe2900af8a0 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe295b48640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901f74f0 con 0x7fe29006b700 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe295b48640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901f77d0 con 0x7fe2900afc80 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 0x7fe29014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 0x7fe29014e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35166/0 (socket says 172.21.15.116:35166) 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 -- 172.21.15.116:0/2137812206 learned_addr learned my addr 172.21.15.116:0/2137812206 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28f7fe640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe29006b700 0x7fe2900b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28ffff640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900afc80 0x7fe2900b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 -- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900afc80 msgr2=0x7fe2900b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900afc80 0x7fe2900b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.786 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 -- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe29006b700 msgr2=0x7fe2900b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe29006b700 0x7fe2900b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 -- 172.21.15.116:0/2137812206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe280009400 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.791+0000 7fe28effd640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 0x7fe29014e5c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e870 tx=0x7fe28400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.792+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2840094f0 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.792+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28400c940 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.792+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28400ccf0 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.792+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2901f7ab0 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.792+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29009e570 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.793+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2900b2610 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.793+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe284013470 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.793+0000 7fe277fff640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe27c06bc70 0x7fe27c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.793+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe28408ac70 con 0x7fe2900af8a0 2024-02-18T11:48:31.787 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.793+0000 7fe28f7fe640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe27c06bc70 0x7fe27c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:31.788 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.794+0000 7fe28f7fe640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe27c06bc70 0x7fe27c06e130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe280036000 tx=0x7fe28003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:31.790 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.798+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe284055060 con 0x7fe2900af8a0 2024-02-18T11:48:31.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.968+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe290074520 con 0x7fe2900af8a0 2024-02-18T11:48:31.961 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.968+0000 7fe277fff640 1 -- 172.21.15.116:0/2137812206 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe284058720 con 0x7fe2900af8a0 2024-02-18T11:48:31.961 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe27c06bc70 msgr2=0x7fe27c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe27c06bc70 0x7fe27c06e130 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe280036000 tx=0x7fe28003c040 comp rx=0 tx=0).stop 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 msgr2=0x7fe29014e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 0x7fe29014e5c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e870 tx=0x7fe28400ed30 comp rx=0 tx=0).stop 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 shutdown_connections 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe27c06bc70 0x7fe27c06e130 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe2900afc80 0x7fe2900b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900af8a0 0x7fe29014e5c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 --2- 172.21.15.116:0/2137812206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe29006b700 0x7fe2900b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 >> 172.21.15.116:0/2137812206 conn(0x7fe2900b2230 msgr2=0x7fe29015ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:31.963 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 shutdown_connections 2024-02-18T11:48:31.964 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:31.970+0000 7fe295b48640 1 -- 172.21.15.116:0/2137812206 wait complete. 2024-02-18T11:48:31.974 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T11:48:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.082+0000 7f9a9174b640 1 Processor -- start 2024-02-18T11:48:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.082+0000 7f9a9174b640 1 -- start start 2024-02-18T11:48:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a9174b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a8c15dbf0 0x7f9a8c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a9174b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a8c0b9370 0x7f9a8c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a9174b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c0cf0d0 con 0x7f9a8c0cec80 2024-02-18T11:48:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a9174b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c0d1ee0 con 0x7f9a8c0b9370 2024-02-18T11:48:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a9174b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c161dc0 con 0x7f9a8c15dbf0 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a8c0b9370 0x7f9a8c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a8c0b9370 0x7f9a8c0ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35176/0 (socket says 172.21.15.116:35176) 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 -- 172.21.15.116:0/3970594973 learned_addr learned my addr 172.21.15.116:0/3970594973 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8affd640 1 --2- 172.21.15.116:0/3970594973 >> v2:172.21.15.143:3300/0 conn(0x7f9a8c15dbf0 0x7f9a8c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 -- 172.21.15.116:0/3970594973 >> v2:172.21.15.143:3300/0 conn(0x7f9a8c15dbf0 msgr2=0x7f9a8c1617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 --2- 172.21.15.116:0/3970594973 >> v2:172.21.15.143:3300/0 conn(0x7f9a8c15dbf0 0x7f9a8c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 -- 172.21.15.116:0/3970594973 >> v1:172.21.15.143:6789/0 conn(0x7f9a8c0cec80 legacy=0x7f9a8c06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.083+0000 7f9a8a7fc640 1 -- 172.21.15.116:0/3970594973 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c1620a0 con 0x7f9a8c0b9370 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.084+0000 7f9a8a7fc640 1 --2- 172.21.15.116:0/3970594973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c0ce740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06d330 tx=0x7f9a8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737a5f17000a0058 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.084+0000 7f9a89ffb640 1 -- 172.21.15.116:0/3970594973 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80031870 con 0x7f9a8c0b9370 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.084+0000 7f9a89ffb640 1 -- 172.21.15.116:0/3970594973 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a800319d0 con 0x7f9a8c0b9370 2024-02-18T11:48:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.084+0000 7f9a89ffb640 1 -- 172.21.15.116:0/3970594973 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a80031d60 con 0x7f9a8c0b9370 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 -- 172.21.15.116:0/3970594973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 msgr2=0x7f9a8c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3970594973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c0ce740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c06d330 tx=0x7f9a8002b910 comp rx=0 tx=0).stop 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 -- 172.21.15.116:0/3970594973 shutdown_connections 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3970594973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c0ce740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3970594973 >> v2:172.21.15.143:3300/0 conn(0x7f9a8c15dbf0 0x7f9a8c1617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 -- 172.21.15.116:0/3970594973 >> 172.21.15.116:0/3970594973 conn(0x7f9a8c0b5480 msgr2=0x7f9a8c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 -- 172.21.15.116:0/3970594973 shutdown_connections 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.085+0000 7f9a9174b640 1 -- 172.21.15.116:0/3970594973 wait complete. 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.086+0000 7f9a9174b640 1 Processor -- start 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.086+0000 7f9a9174b640 1 -- start start 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a8c0cec80 0x7f9a8c160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8c15dbf0 0x7f9a8c1f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c154700 con 0x7f9a8c0b9370 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c154b00 con 0x7f9a8c15dbf0 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c153a80 con 0x7f9a8c0cec80 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35188/0 (socket says 172.21.15.116:35188) 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 -- 172.21.15.116:0/3319074335 learned_addr learned my addr 172.21.15.116:0/3319074335 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 -- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a8c0cec80 msgr2=0x7f9a8c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a8c0cec80 0x7f9a8c160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8b7fe640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8c15dbf0 0x7f9a8c1f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 -- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8c15dbf0 msgr2=0x7f9a8c1f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8c15dbf0 0x7f9a8c1f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 -- 172.21.15.116:0/3319074335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a800094d0 con 0x7f9a8c0b9370 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8b7fe640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8c15dbf0 0x7f9a8c1f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a8affd640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c155970 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00c7f0 tx=0x7f9a7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c004220 con 0x7f9a8c0b9370 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c004380 con 0x7f9a8c0b9370 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c15e150 con 0x7f9a8c0b9370 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.087+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c007840 con 0x7f9a8c0b9370 2024-02-18T11:48:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.088+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c154040 con 0x7f9a8c0b9370 2024-02-18T11:48:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.088+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c067840 con 0x7f9a8c0b9370 2024-02-18T11:48:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.089+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9a7c007bb0 con 0x7f9a8c0b9370 2024-02-18T11:48:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.089+0000 7f9a737fe640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a7806bc70 0x7f9a7806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.090+0000 7f9a8a7fc640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a7806bc70 0x7f9a7806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.090+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9a7c08aa50 con 0x7f9a8c0b9370 2024-02-18T11:48:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.090+0000 7f9a8a7fc640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a7806bc70 0x7f9a7806e130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a80036000 tx=0x7f9a80030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.086 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.094+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a7c054ea0 con 0x7f9a8c0b9370 2024-02-18T11:48:32.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.277+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a8c074520 con 0x7f9a8c0b9370 2024-02-18T11:48:32.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.277+0000 7f9a737fe640 1 -- 172.21.15.116:0/3319074335 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9a7c007e60 con 0x7f9a8c0b9370 2024-02-18T11:48:32.269 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a7806bc70 msgr2=0x7f9a7806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a7806bc70 0x7f9a7806e130 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9a80036000 tx=0x7f9a80030000 comp rx=0 tx=0).stop 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 msgr2=0x7f9a8c155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c155970 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00c7f0 tx=0x7f9a7c00ccb0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 shutdown_connections 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f9a7806bc70 0x7f9a7806e130 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a8c15dbf0 0x7f9a8c1f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9a8c0cec80 0x7f9a8c160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 --2- 172.21.15.116:0/3319074335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a8c0b9370 0x7f9a8c155970 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 >> 172.21.15.116:0/3319074335 conn(0x7f9a8c0b5480 msgr2=0x7f9a8c0b8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:32.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 shutdown_connections 2024-02-18T11:48:32.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.279+0000 7f9a9174b640 1 -- 172.21.15.116:0/3319074335 wait complete. 2024-02-18T11:48:32.282 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-02-18T11:48:32.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:32.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-02-18T11:48:32.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T11:48:32.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:32.316 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-02-18T11:48:32.373 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-02-18T11:48:32.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:32.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.481+0000 7f161880e640 1 Processor -- start 2024-02-18T11:48:32.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.481+0000 7f161880e640 1 -- start start 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.482+0000 7f161880e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f161415a860 0x7f16140cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.482+0000 7f161880e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140b02e0 con 0x7f16140cd1f0 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.482+0000 7f161880e640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16140af6d0 con 0x7f161406ea20 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.482+0000 7f161880e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16140af9b0 con 0x7f161415a860 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f161259c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f161415a860 0x7f16140cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f1611d9b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f16140cd1f0 0x7f161406e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55634/0 (socket says 172.21.15.116:55634) 2024-02-18T11:48:32.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f1611d9b640 1 -- 172.21.15.116:0/206076042 learned_addr learned my addr 172.21.15.116:0/206076042 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:32.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f161259c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f161415a860 0x7f16140cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35198/0 (socket says 172.21.15.116:35198) 2024-02-18T11:48:32.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f161259c640 1 -- 172.21.15.116:0/206076042 >> v1:172.21.15.143:6789/0 conn(0x7f161406ea20 legacy=0x7f16140b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:32.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f161259c640 1 -- 172.21.15.116:0/206076042 >> v1:172.21.15.116:6789/0 conn(0x7f16140cd1f0 legacy=0x7f161406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f161159a640 1 -- 172.21.15.116:0/206076042 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413411918 0 0) 0x7f16140b02e0 con 0x7f16140cd1f0 2024-02-18T11:48:32.476 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.483+0000 7f161259c640 1 -- 172.21.15.116:0/206076042 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140afc90 con 0x7f161415a860 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161259c640 1 --2- 172.21.15.116:0/206076042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f161415a860 0x7f16140cccb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f160c009d90 tx=0x7f160c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15af0e0ad8ca0813 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161159a640 1 -- 172.21.15.116:0/206076042 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c02cc60 con 0x7f161415a860 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161159a640 1 -- 172.21.15.116:0/206076042 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c02cdc0 con 0x7f161415a860 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161159a640 1 -- 172.21.15.116:0/206076042 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c034660 con 0x7f161415a860 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 -- 172.21.15.116:0/206076042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f161415a860 msgr2=0x7f16140cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 --2- 172.21.15.116:0/206076042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f161415a860 0x7f16140cccb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f160c009d90 tx=0x7f160c02beb0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.477 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 -- 172.21.15.116:0/206076042 shutdown_connections 2024-02-18T11:48:32.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 --2- 172.21.15.116:0/206076042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f161415a860 0x7f16140cccb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 -- 172.21.15.116:0/206076042 >> 172.21.15.116:0/206076042 conn(0x7f1614066f20 msgr2=0x7f161406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:32.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 -- 172.21.15.116:0/206076042 shutdown_connections 2024-02-18T11:48:32.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.484+0000 7f161880e640 1 -- 172.21.15.116:0/206076042 wait complete. 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 Processor -- start 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 -- start start 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f161406ea20 0x7f16141f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 0x7f16141466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f161415a860 0x7f16141f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f4800 con 0x7f16140cd1f0 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1614145f50 con 0x7f161415a860 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f161880e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16141f4a30 con 0x7f161406ea20 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.486+0000 7f1611d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 0x7f16141466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 0x7f16141466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35210/0 (socket says 172.21.15.116:35210) 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 -- 172.21.15.116:0/621090951 learned_addr learned my addr 172.21.15.116:0/621090951 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 -- 172.21.15.116:0/621090951 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f161406ea20 msgr2=0x7f16141f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f161406ea20 0x7f16141f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 -- 172.21.15.116:0/621090951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f161415a860 msgr2=0x7f16141f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f161415a860 0x7f16141f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 -- 172.21.15.116:0/621090951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c009a70 con 0x7f16140cd1f0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1611d9b640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 0x7f16141466c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f160800c4b0 tx=0x7f160800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16080074e0 con 0x7f16140cd1f0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1608007660 con 0x7f16140cd1f0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16141f4d10 con 0x7f16140cd1f0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608004a20 con 0x7f16140cd1f0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.487+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16141461a0 con 0x7f16140cd1f0 2024-02-18T11:48:32.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.488+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16141463c0 con 0x7f16140cd1f0 2024-02-18T11:48:32.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.488+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f1608004b80 con 0x7f16140cd1f0 2024-02-18T11:48:32.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.489+0000 7f1602ffd640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f406bc70 0x7f15f406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.489+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f160808a010 con 0x7f16140cd1f0 2024-02-18T11:48:32.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.489+0000 7f161259c640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f406bc70 0x7f15f406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.489+0000 7f161259c640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f406bc70 0x7f15f406e130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f160c036000 tx=0x7f160c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.493+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1608054520 con 0x7f16140cd1f0 2024-02-18T11:48:32.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.698+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1614074520 con 0x7f16140cd1f0 2024-02-18T11:48:32.691 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.699+0000 7f1602ffd640 1 -- 172.21.15.116:0/621090951 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f1608018090 con 0x7f16140cd1f0 2024-02-18T11:48:32.691 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:32.691 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:32.691 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f406bc70 msgr2=0x7f15f406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f406bc70 0x7f15f406e130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f160c036000 tx=0x7f160c02e000 comp rx=0 tx=0).stop 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 msgr2=0x7f16141466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 0x7f16141466c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f160800c4b0 tx=0x7f160800c970 comp rx=0 tx=0).stop 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 shutdown_connections 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f15f406bc70 0x7f15f406e130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f161415a860 0x7f16141f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16140cd1f0 0x7f16141466c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.700+0000 7f161880e640 1 --2- 172.21.15.116:0/621090951 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f161406ea20 0x7f16141f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.701+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 >> 172.21.15.116:0/621090951 conn(0x7f1614066f20 msgr2=0x7f1614157dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:32.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.701+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 shutdown_connections 2024-02-18T11:48:32.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.701+0000 7f161880e640 1 -- 172.21.15.116:0/621090951 wait complete. 2024-02-18T11:48:32.704 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:32.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.812+0000 7f4aa52a3640 1 Processor -- start 2024-02-18T11:48:32.804 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.812+0000 7f4aa52a3640 1 -- start start 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4aa52a3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4aa01611e0 0x7f4aa00b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4aa52a3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4aa00cea00 0x7f4aa006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4aa52a3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa00cee60 0x7f4aa006c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4aa52a3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa00b7030 con 0x7f4aa00cea00 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4aa52a3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa00d1fa0 con 0x7f4aa01611e0 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4aa52a3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00d2560 con 0x7f4aa00cee60 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4aa00cea00 0x7f4aa006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.806 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4aa00cea00 0x7f4aa006c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35222/0 (socket says 172.21.15.116:35222) 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1658970710 learned_addr learned my addr 172.21.15.116:0/1658970710 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1658970710 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa00cee60 msgr2=0x7f4aa006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 --2- 172.21.15.116:0/1658970710 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa00cee60 0x7f4aa006c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1658970710 >> v2:172.21.15.143:3300/0 conn(0x7f4aa01611e0 msgr2=0x7f4aa00b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 --2- 172.21.15.116:0/1658970710 >> v2:172.21.15.143:3300/0 conn(0x7f4aa01611e0 0x7f4aa00b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.813+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1658970710 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa00d4740 con 0x7f4aa00cea00 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.814+0000 7f4a9e7fc640 1 --2- 172.21.15.116:0/1658970710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cea00 0x7f4aa006c070 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4aa008c420 tx=0x7f4a9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43452337d30635a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.814+0000 7f4a9dffb640 1 -- 172.21.15.116:0/1658970710 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94031870 con 0x7f4aa00cea00 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.814+0000 7f4a9dffb640 1 -- 172.21.15.116:0/1658970710 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a940319d0 con 0x7f4aa00cea00 2024-02-18T11:48:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.814+0000 7f4a9dffb640 1 -- 172.21.15.116:0/1658970710 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94031d60 con 0x7f4aa00cea00 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.814+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1658970710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cea00 msgr2=0x7f4aa006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.814+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1658970710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cea00 0x7f4aa006c070 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4aa008c420 tx=0x7f4a9402b910 comp rx=0 tx=0).stop 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1658970710 shutdown_connections 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1658970710 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa00cee60 0x7f4aa006c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1658970710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cea00 0x7f4aa006c070 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1658970710 >> v2:172.21.15.143:3300/0 conn(0x7f4aa01611e0 0x7f4aa00b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1658970710 >> 172.21.15.116:0/1658970710 conn(0x7f4aa00b5b00 msgr2=0x7f4aa00d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1658970710 shutdown_connections 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.815+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1658970710 wait complete. 2024-02-18T11:48:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.816+0000 7f4aa52a3640 1 Processor -- start 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.816+0000 7f4aa52a3640 1 -- start start 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4aa00cea00 0x7f4aa01f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 0x7f4aa015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa01611e0 0x7f4aa0157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00c8630 con 0x7f4aa00cee60 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00c8910 con 0x7f4aa00cea00 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa00c8bf0 con 0x7f4aa01611e0 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 0x7f4aa015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 0x7f4aa015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35232/0 (socket says 172.21.15.116:35232) 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1234496148 learned_addr learned my addr 172.21.15.116:0/1234496148 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9f7fe640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa01611e0 0x7f4aa0157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9effd640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4aa00cea00 0x7f4aa01f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa01611e0 msgr2=0x7f4aa0157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa01611e0 0x7f4aa0157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4aa00cea00 msgr2=0x7f4aa01f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4aa00cea00 0x7f4aa01f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 -- 172.21.15.116:0/1234496148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a940094d0 con 0x7f4aa00cee60 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9f7fe640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa01611e0 0x7f4aa0157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9effd640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4aa00cea00 0x7f4aa01f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a9e7fc640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 0x7f4aa015d6f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4a94000c00 tx=0x7f4a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9403ad30 con 0x7f4aa00cee60 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9403ae90 con 0x7f4aa00cee60 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94044610 con 0x7f4aa00cee60 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa00c8ed0 con 0x7f4aa00cee60 2024-02-18T11:48:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.817+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa015dc30 con 0x7f4aa00cee60 2024-02-18T11:48:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.818+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa0067840 con 0x7f4aa00cee60 2024-02-18T11:48:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.819+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4a9404d450 con 0x7f4aa00cee60 2024-02-18T11:48:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.819+0000 7f4a877fe640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4a8c06bc70 0x7f4a8c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.819+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f4a9402d040 con 0x7f4aa00cee60 2024-02-18T11:48:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.819+0000 7f4a9effd640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4a8c06bc70 0x7f4a8c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:32.813 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.820+0000 7f4a9effd640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4a8c06bc70 0x7f4a8c06e130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4aa01f6fe0 tx=0x7f4a900073e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:32.815 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.823+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a94077650 con 0x7f4aa00cee60 2024-02-18T11:48:32.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.997+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4aa00c93f0 con 0x7f4aa00cee60 2024-02-18T11:48:32.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:32.998+0000 7f4a877fe640 1 -- 172.21.15.116:0/1234496148 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f4aa00c93f0 con 0x7f4aa00cee60 2024-02-18T11:48:32.991 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:32.991 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4a8c06bc70 msgr2=0x7f4a8c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4a8c06bc70 0x7f4a8c06e130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4aa01f6fe0 tx=0x7f4a900073e0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 msgr2=0x7f4aa015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 0x7f4aa015d6f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4a94000c00 tx=0x7f4a940046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 shutdown_connections 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4a8c06bc70 0x7f4a8c06e130 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4aa01611e0 0x7f4aa0157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4aa00cee60 0x7f4aa015d6f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 --2- 172.21.15.116:0/1234496148 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4aa00cea00 0x7f4aa01f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 >> 172.21.15.116:0/1234496148 conn(0x7f4aa00b5b00 msgr2=0x7f4aa00d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 shutdown_connections 2024-02-18T11:48:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.000+0000 7f4aa52a3640 1 -- 172.21.15.116:0/1234496148 wait complete. 2024-02-18T11:48:33.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:47:32.328152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:47:32.824865+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-18T11:47:36.561926+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-18T11:48:33.004 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:33.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.115+0000 7f90cad8a640 1 Processor -- start 2024-02-18T11:48:33.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.115+0000 7f90cad8a640 1 -- start start 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90cad8a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90c4150ed0 0x7f90c40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90cad8a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90c4072160 0x7f90c415da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90cad8a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c415a480 con 0x7f90c40cdd20 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90cad8a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c415a760 con 0x7f90c4072160 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90cad8a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c415aa40 con 0x7f90c4150ed0 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90c4072160 0x7f90c415da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90c4072160 0x7f90c415da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35240/0 (socket says 172.21.15.116:35240) 2024-02-18T11:48:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 -- 172.21.15.116:0/760045748 learned_addr learned my addr 172.21.15.116:0/760045748 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c8b26640 1 --2- 172.21.15.116:0/760045748 >> v2:172.21.15.143:3300/0 conn(0x7f90c4150ed0 0x7f90c40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 -- 172.21.15.116:0/760045748 >> v2:172.21.15.143:3300/0 conn(0x7f90c4150ed0 msgr2=0x7f90c40b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 --2- 172.21.15.116:0/760045748 >> v2:172.21.15.143:3300/0 conn(0x7f90c4150ed0 0x7f90c40b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 -- 172.21.15.116:0/760045748 >> v1:172.21.15.143:6789/0 conn(0x7f90c40cdd20 legacy=0x7f90c4071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.116+0000 7f90c9327640 1 -- 172.21.15.116:0/760045748 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c415ad20 con 0x7f90c4072160 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.117+0000 7f90c9327640 1 --2- 172.21.15.116:0/760045748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4072160 0x7f90c415da70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d150 tx=0x7f90bc02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b089f63e20e5aa32 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.117+0000 7f90c37fe640 1 -- 172.21.15.116:0/760045748 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc02ea80 con 0x7f90c4072160 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.117+0000 7f90c37fe640 1 -- 172.21.15.116:0/760045748 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc02ebe0 con 0x7f90c4072160 2024-02-18T11:48:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.117+0000 7f90c37fe640 1 -- 172.21.15.116:0/760045748 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc009600 con 0x7f90c4072160 2024-02-18T11:48:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 -- 172.21.15.116:0/760045748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4072160 msgr2=0x7f90c415da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 --2- 172.21.15.116:0/760045748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4072160 0x7f90c415da70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d150 tx=0x7f90bc02e0e0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 -- 172.21.15.116:0/760045748 shutdown_connections 2024-02-18T11:48:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 --2- 172.21.15.116:0/760045748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4072160 0x7f90c415da70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 --2- 172.21.15.116:0/760045748 >> v2:172.21.15.143:3300/0 conn(0x7f90c4150ed0 0x7f90c40b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 -- 172.21.15.116:0/760045748 >> 172.21.15.116:0/760045748 conn(0x7f90c4067000 msgr2=0x7f90c406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:33.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 -- 172.21.15.116:0/760045748 shutdown_connections 2024-02-18T11:48:33.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.118+0000 7f90cad8a640 1 -- 172.21.15.116:0/760045748 wait complete. 2024-02-18T11:48:33.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.119+0000 7f90cad8a640 1 Processor -- start 2024-02-18T11:48:33.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.119+0000 7f90cad8a640 1 -- start start 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c4072160 0x7f90c415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90c40cdd20 0x7f90c415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 0x7f90c415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c4145080 con 0x7f90c4150ed0 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c415bb90 con 0x7f90c4072160 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c4144750 con 0x7f90c40cdd20 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 0x7f90c415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 0x7f90c415c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35246/0 (socket says 172.21.15.116:35246) 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 -- 172.21.15.116:0/2732076619 learned_addr learned my addr 172.21.15.116:0/2732076619 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 -- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90c40cdd20 msgr2=0x7f90c415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90c40cdd20 0x7f90c415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.114 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 -- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c4072160 msgr2=0x7f90c415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c8b26640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c4072160 0x7f90c415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c4072160 0x7f90c415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 -- 172.21.15.116:0/2732076619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc00bd40 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c8b26640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c4072160 0x7f90c415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c9327640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 0x7f90c415c130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d150 tx=0x7f90bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc02ea80 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90bc00b720 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4144a30 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0408a0 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.120+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c4147830 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.121+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c4145230 con 0x7f90c4150ed0 2024-02-18T11:48:33.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.122+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f90bc02ed50 con 0x7f90c4150ed0 2024-02-18T11:48:33.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.122+0000 7f90c17fa640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90b006bc70 0x7f90b006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.122+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f90bc0af8b0 con 0x7f90c4150ed0 2024-02-18T11:48:33.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.122+0000 7f90c8b26640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90b006bc70 0x7f90b006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.123+0000 7f90c8b26640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90b006bc70 0x7f90b006e130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006fd0 tx=0x7f90b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.126+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90c4145230 con 0x7f90c4150ed0 2024-02-18T11:48:33.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.296+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90c4074520 con 0x7f90c4150ed0 2024-02-18T11:48:33.288 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.296+0000 7f90c17fa640 1 -- 172.21.15.116:0/2732076619 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90bc079c80 con 0x7f90c4150ed0 2024-02-18T11:48:33.289 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:33.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90b006bc70 msgr2=0x7f90b006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90b006bc70 0x7f90b006e130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f90b4006fd0 tx=0x7f90b4008040 comp rx=0 tx=0).stop 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 msgr2=0x7f90c415c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 0x7f90c415c130 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00d150 tx=0x7f90bc004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 shutdown_connections 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f90b006bc70 0x7f90b006e130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4150ed0 0x7f90c415c130 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90c40cdd20 0x7f90c415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 --2- 172.21.15.116:0/2732076619 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c4072160 0x7f90c415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 >> 172.21.15.116:0/2732076619 conn(0x7f90c4067000 msgr2=0x7f90c40afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 shutdown_connections 2024-02-18T11:48:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.298+0000 7f90cad8a640 1 -- 172.21.15.116:0/2732076619 wait complete. 2024-02-18T11:48:33.301 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T11:48:33.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.412+0000 7facc8064640 1 Processor -- start 2024-02-18T11:48:33.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.412+0000 7facc8064640 1 -- start start 2024-02-18T11:48:33.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc8064640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc015dba0 0x7facc0161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc8064640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7facc00b93b0 0x7facc00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc8064640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc00cf180 con 0x7facc00ced30 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc8064640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc00d21b0 con 0x7facc00b93b0 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc8064640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00d1e60 con 0x7facc015dba0 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc6601640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7facc00ced30 0x7facc006c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55642/0 (socket says 172.21.15.116:55642) 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc6601640 1 -- 172.21.15.116:0/650835004 learned_addr learned my addr 172.21.15.116:0/650835004 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc55ff640 1 --2- 172.21.15.116:0/650835004 >> v2:172.21.15.143:3300/0 conn(0x7facc00b93b0 0x7facc00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc5e00640 1 --2- 172.21.15.116:0/650835004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc015dba0 0x7facc0161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc4dfe640 1 -- 172.21.15.116:0/650835004 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089275003 0 0) 0x7facb8001110 con 0x7facc00ced30 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc4dfe640 1 -- 172.21.15.116:0/650835004 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facac003520 con 0x7facc00ced30 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc5e00640 1 -- 172.21.15.116:0/650835004 >> v2:172.21.15.143:3300/0 conn(0x7facc00b93b0 msgr2=0x7facc00ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc5e00640 1 --2- 172.21.15.116:0/650835004 >> v2:172.21.15.143:3300/0 conn(0x7facc00b93b0 0x7facc00ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc5e00640 1 -- 172.21.15.116:0/650835004 >> v1:172.21.15.116:6789/0 conn(0x7facc00ced30 legacy=0x7facc006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc5e00640 1 -- 172.21.15.116:0/650835004 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0162050 con 0x7facc015dba0 2024-02-18T11:48:33.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.413+0000 7facc55ff640 1 --2- 172.21.15.116:0/650835004 >> v2:172.21.15.143:3300/0 conn(0x7facc00b93b0 0x7facc00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.414+0000 7facc5e00640 1 --2- 172.21.15.116:0/650835004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc015dba0 0x7facc0161800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7facbc009d90 tx=0x7facbc02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17b1493c91f789e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.414+0000 7facc4dfe640 1 -- 172.21.15.116:0/650835004 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc02cc50 con 0x7facc015dba0 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.414+0000 7facc4dfe640 1 -- 172.21.15.116:0/650835004 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc02cdb0 con 0x7facc015dba0 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.414+0000 7facc4dfe640 1 -- 172.21.15.116:0/650835004 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc034640 con 0x7facc015dba0 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 -- 172.21.15.116:0/650835004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc015dba0 msgr2=0x7facc0161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 --2- 172.21.15.116:0/650835004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc015dba0 0x7facc0161800 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7facbc009d90 tx=0x7facbc02bea0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 -- 172.21.15.116:0/650835004 shutdown_connections 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 --2- 172.21.15.116:0/650835004 >> v2:172.21.15.143:3300/0 conn(0x7facc00b93b0 0x7facc00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 --2- 172.21.15.116:0/650835004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc015dba0 0x7facc0161800 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 -- 172.21.15.116:0/650835004 >> 172.21.15.116:0/650835004 conn(0x7facc00d04a0 msgr2=0x7facc00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 -- 172.21.15.116:0/650835004 shutdown_connections 2024-02-18T11:48:33.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.415+0000 7facc8064640 1 -- 172.21.15.116:0/650835004 wait complete. 2024-02-18T11:48:33.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.416+0000 7facc8064640 1 Processor -- start 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.416+0000 7facc8064640 1 -- start start 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc8064640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 0x7facc00c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc8064640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc00ced30 0x7facc00d2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc8064640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc015dba0 0x7facc00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc8064640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01f7e40 con 0x7facc00b93b0 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc8064640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc01f71c0 con 0x7facc015dba0 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc8064640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc01f74a0 con 0x7facc00ced30 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc5e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 0x7facc00c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc5e00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 0x7facc00c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35248/0 (socket says 172.21.15.116:35248) 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc5e00640 1 -- 172.21.15.116:0/724072477 learned_addr learned my addr 172.21.15.116:0/724072477 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.417+0000 7facc55ff640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc00ced30 0x7facc00d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc6601640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc015dba0 0x7facc00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc5e00640 1 -- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc00ced30 msgr2=0x7facc00d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc5e00640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc00ced30 0x7facc00d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc5e00640 1 -- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc015dba0 msgr2=0x7facc00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc5e00640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc015dba0 0x7facc00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc5e00640 1 -- 172.21.15.116:0/724072477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb0009400 con 0x7facc00b93b0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc6601640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc015dba0 0x7facc00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc5e00640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 0x7facc00c9380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7facbc000f80 tx=0x7facbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc004880 con 0x7facc00b93b0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc009450 con 0x7facc00b93b0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc009a70 con 0x7facc00b93b0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc034660 con 0x7facc00b93b0 2024-02-18T11:48:33.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.418+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc01f7a60 con 0x7facc00b93b0 2024-02-18T11:48:33.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.419+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc01f7f80 con 0x7facc00b93b0 2024-02-18T11:48:33.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.419+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7facbc0347c0 con 0x7facc00b93b0 2024-02-18T11:48:33.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.420+0000 7facb67fc640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faca406bc70 0x7faca406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.420+0000 7facc55ff640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faca406bc70 0x7faca406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.420+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7facbc0ad990 con 0x7facc00b93b0 2024-02-18T11:48:33.414 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.420+0000 7facc55ff640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faca406bc70 0x7faca406e130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7facc00d3100 tx=0x7facb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.424+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7facc01f7f80 con 0x7facc00b93b0 2024-02-18T11:48:33.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.593+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7facc01f7f80 con 0x7facc00b93b0 2024-02-18T11:48:33.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.594+0000 7facb67fc640 1 -- 172.21.15.116:0/724072477 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7facc01f7f80 con 0x7facc00b93b0 2024-02-18T11:48:33.586 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:33.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faca406bc70 msgr2=0x7faca406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faca406bc70 0x7faca406e130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7facc00d3100 tx=0x7facb0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:33.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 msgr2=0x7facc00c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 0x7facc00c9380 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7facbc000f80 tx=0x7facbc004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:33.588 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 shutdown_connections 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faca406bc70 0x7faca406e130 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc015dba0 0x7facc00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7facc00ced30 0x7facc00d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 --2- 172.21.15.116:0/724072477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facc00b93b0 0x7facc00c9380 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 >> 172.21.15.116:0/724072477 conn(0x7facc00d04a0 msgr2=0x7facc00b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 shutdown_connections 2024-02-18T11:48:33.589 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.596+0000 7facc8064640 1 -- 172.21.15.116:0/724072477 wait complete. 2024-02-18T11:48:33.599 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T11:48:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.708+0000 7fdaf201b640 1 Processor -- start 2024-02-18T11:48:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.708+0000 7fdaf201b640 1 -- start start 2024-02-18T11:48:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaf201b640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdaec06ea40 0x7fdaec15e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaf201b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaec0b3610 con 0x7fdaec144090 2024-02-18T11:48:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaf201b640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaec0b2a50 con 0x7fdaec0b31a0 2024-02-18T11:48:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaf201b640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaec0cd0f0 con 0x7fdaec06ea40 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaeb7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdaec144090 0x7fdaec0cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55648/0 (socket says 172.21.15.116:55648) 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaeb7fe640 1 -- 172.21.15.116:0/463898070 learned_addr learned my addr 172.21.15.116:0/463898070 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529893183 0 0) 0x7fdaec0b3610 con 0x7fdaec144090 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad4003520 con 0x7fdaec144090 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.709+0000 7fdaebfff640 1 --2- 172.21.15.116:0/463898070 >> v2:172.21.15.143:3300/0 conn(0x7fdaec06ea40 0x7fdaec15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3919301790 0 0) 0x7fdad4003520 con 0x7fdaec144090 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 >> v2:172.21.15.143:3300/0 conn(0x7fdaec06ea40 msgr2=0x7fdaec15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 --2- 172.21.15.116:0/463898070 >> v2:172.21.15.143:3300/0 conn(0x7fdaec06ea40 0x7fdaec15e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 >> v1:172.21.15.143:6789/0 conn(0x7fdaec0b31a0 legacy=0x7fdaec06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec15b090 con 0x7fdaec144090 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fdadc001670 con 0x7fdaec144090 2024-02-18T11:48:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 >> v1:172.21.15.116:6789/0 conn(0x7fdaec144090 legacy=0x7fdaec0cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdad40039c0 0x7fdad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdad40047c0 0x7fdad4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40051d0 0x7fdad400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec0cd0f0 con 0x7fdad40047c0 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad400c120 con 0x7fdad40051d0 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad400c2e0 con 0x7fdad40039c0 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdad40047c0 0x7fdad4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeb7fe640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdad40039c0 0x7fdad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaebfff640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40051d0 0x7fdad400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeaffd640 1 -- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdad40039c0 msgr2=0x7fdad4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdad40039c0 0x7fdad4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeaffd640 1 -- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40051d0 msgr2=0x7fdad400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40051d0 0x7fdad400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaebfff640 1 -- 172.21.15.116:0/463898070 reap_dead start 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.710+0000 7fdaeaffd640 1 -- 172.21.15.116:0/463898070 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4003520 con 0x7fdad40047c0 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdad40047c0 0x7fdad4004c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdae0001330 tx=0x7fdad40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b00dd3e7f8e822 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00052d0 con 0x7fdad40047c0 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0005430 con 0x7fdad40047c0 2024-02-18T11:48:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaea7fc640 1 -- 172.21.15.116:0/463898070 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0003830 con 0x7fdad40047c0 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaf201b640 1 -- 172.21.15.116:0/463898070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdad40047c0 msgr2=0x7fdad4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaf201b640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdad40047c0 0x7fdad4004c40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdae0001330 tx=0x7fdad40057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaf201b640 1 -- 172.21.15.116:0/463898070 shutdown_connections 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaf201b640 1 --2- 172.21.15.116:0/463898070 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdad40047c0 0x7fdad4004c40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaf201b640 1 -- 172.21.15.116:0/463898070 >> 172.21.15.116:0/463898070 conn(0x7fdaec067000 msgr2=0x7fdaec06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.711+0000 7fdaf201b640 1 -- 172.21.15.116:0/463898070 shutdown_connections 2024-02-18T11:48:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.712+0000 7fdaf201b640 1 -- 172.21.15.116:0/463898070 wait complete. 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.713+0000 7fdaf201b640 1 Processor -- start 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.713+0000 7fdaf201b640 1 -- start start 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.713+0000 7fdaf201b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40047c0 0x7fdaec13aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 0x7fdaec06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdaec1ef5c0 0x7fdaec1efa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec06dcc0 con 0x7fdaec06d300 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec13ce70 con 0x7fdad40047c0 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec13c260 con 0x7fdaec1ef5c0 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 0x7fdaec06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 0x7fdaec06d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35256/0 (socket says 172.21.15.116:35256) 2024-02-18T11:48:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 -- 172.21.15.116:0/436429961 learned_addr learned my addr 172.21.15.116:0/436429961 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 -- 172.21.15.116:0/436429961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdaec1ef5c0 msgr2=0x7fdaec1efa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdaec1ef5c0 0x7fdaec1efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 -- 172.21.15.116:0/436429961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40047c0 msgr2=0x7fdaec13aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40047c0 0x7fdaec13aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 -- 172.21.15.116:0/436429961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec13c540 con 0x7fdaec06d300 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaeaffd640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 0x7fdaec06d780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdae0001810 tx=0x7fdae0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0006be0 con 0x7fdaec06d300 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0006d40 con 0x7fdaec06d300 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec13c820 con 0x7fdaec06d300 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0003810 con 0x7fdaec06d300 2024-02-18T11:48:33.712 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.714+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec13b420 con 0x7fdaec06d300 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.715+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec0cbe40 con 0x7fdaec06d300 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.716+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fdae0003970 con 0x7fdaec06d300 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.716+0000 7fdad3fff640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdad806bca0 0x7fdad806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.716+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fdae00ae690 con 0x7fdaec06d300 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.716+0000 7fdaeb7fe640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdad806bca0 0x7fdad806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.717+0000 7fdaeb7fe640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdad806bca0 0x7fdad806e160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdaec06a410 tx=0x7fdadc000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:33.713 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.720+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdae0078b90 con 0x7fdaec06d300 2024-02-18T11:48:33.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.892+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdaec0cd990 con 0x7fdaec06d300 2024-02-18T11:48:33.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.893+0000 7fdad3fff640 1 -- 172.21.15.116:0/436429961 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdae0007a30 con 0x7fdaec06d300 2024-02-18T11:48:33.885 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdad806bca0 msgr2=0x7fdad806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdad806bca0 0x7fdad806e160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdaec06a410 tx=0x7fdadc000ed0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 msgr2=0x7fdaec06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 0x7fdaec06d780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdae0001810 tx=0x7fdae0001120 comp rx=0 tx=0).stop 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 shutdown_connections 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fdad806bca0 0x7fdad806e160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdaec1ef5c0 0x7fdaec1efa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.887 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaec06d300 0x7fdaec06d780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 --2- 172.21.15.116:0/436429961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdad40047c0 0x7fdaec13aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:33.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.894+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 >> 172.21.15.116:0/436429961 conn(0x7fdaec067000 msgr2=0x7fdaec0b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:33.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.895+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 shutdown_connections 2024-02-18T11:48:33.888 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:33.895+0000 7fdaf201b640 1 -- 172.21.15.116:0/436429961 wait complete. 2024-02-18T11:48:33.897 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-18T11:48:34.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.009+0000 7f942505c640 1 Processor -- start 2024-02-18T11:48:34.001 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.009+0000 7f942505c640 1 -- start start 2024-02-18T11:48:34.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.009+0000 7f942505c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 0x7f94200b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f942505c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94200cc9e0 0x7f942006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f942505c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94200b2a50 con 0x7f94200cce20 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f942505c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942015b090 con 0x7f94200cc9e0 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f942505c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942015b370 con 0x7f9420144090 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941f59e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f94200cce20 0x7f942015e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55652/0 (socket says 172.21.15.116:55652) 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941f59e640 1 -- 172.21.15.116:0/960772629 learned_addr learned my addr 172.21.15.116:0/960772629 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731236585 0 0) 0x7f94200b2a50 con 0x7f94200cce20 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f940c003520 con 0x7f94200cce20 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941ed9d640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 0x7f94200b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f9417fff640 1 --2- 172.21.15.116:0/960772629 >> v2:172.21.15.143:3300/0 conn(0x7f94200cc9e0 0x7f942006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2167520153 0 0) 0x7f940c003520 con 0x7f94200cce20 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 msgr2=0x7f94200b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 0x7f94200b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.005 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 >> v2:172.21.15.143:3300/0 conn(0x7f94200cc9e0 msgr2=0x7f942006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 --2- 172.21.15.116:0/960772629 >> v2:172.21.15.143:3300/0 conn(0x7f94200cc9e0 0x7f942006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942015b650 con 0x7f94200cce20 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f94180019f0 con 0x7f94200cce20 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 >> v1:172.21.15.116:6789/0 conn(0x7f94200cce20 legacy=0x7f942015e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 0x7f940c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f940c003fa0 0x7f940c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f940c004d00 0x7f940c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942015b090 con 0x7f940c004d00 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942015b370 con 0x7f940c003fa0 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940c00bb40 con 0x7f9420144090 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941f59e640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f940c004d00 0x7f940c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f941ed9d640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 0x7f940c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.010+0000 7f9417fff640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f940c003fa0 0x7f940c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941f59e640 1 -- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 msgr2=0x7f940c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941f59e640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9420144090 0x7f940c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941f59e640 1 -- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f940c003fa0 msgr2=0x7f940c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941f59e640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f940c003fa0 0x7f940c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941f59e640 1 -- 172.21.15.116:0/960772629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c003520 con 0x7f940c004d00 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941f59e640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f940c004d00 0x7f940c005180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f941800a860 tx=0x7f940c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2c98f9552e5370 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.011+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418034630 con 0x7f940c004d00 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941802f070 con 0x7f940c004d00 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f941e59c640 1 -- 172.21.15.116:0/960772629 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9418034a30 con 0x7f940c004d00 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f942505c640 1 -- 172.21.15.116:0/960772629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f940c004d00 msgr2=0x7f940c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f942505c640 1 --2- 172.21.15.116:0/960772629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f940c004d00 0x7f940c005180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f941800a860 tx=0x7f940c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f941f59e640 1 -- 172.21.15.116:0/960772629 reap_dead start 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f942505c640 1 -- 172.21.15.116:0/960772629 shutdown_connections 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f942505c640 1 -- 172.21.15.116:0/960772629 >> 172.21.15.116:0/960772629 conn(0x7f9420067000 msgr2=0x7f942006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f942505c640 1 -- 172.21.15.116:0/960772629 shutdown_connections 2024-02-18T11:48:34.007 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.012+0000 7f942505c640 1 -- 172.21.15.116:0/960772629 wait complete. 2024-02-18T11:48:34.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.013+0000 7f942505c640 1 Processor -- start 2024-02-18T11:48:34.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.013+0000 7f942505c640 1 -- start start 2024-02-18T11:48:34.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f942505c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f942015a150 0x7f94201393b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f942505c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 0x7f942006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.010 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f942505c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f942006dd10 0x7f9420137b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f942505c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942006f1c0 con 0x7f94201398f0 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f942505c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9420139d70 con 0x7f942006dd10 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f942505c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942006e540 con 0x7f942015a150 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 0x7f942006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 0x7f942006d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35262/0 (socket says 172.21.15.116:35262) 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 -- 172.21.15.116:0/2916459308 learned_addr learned my addr 172.21.15.116:0/2916459308 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f941ed9d640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f942015a150 0x7f94201393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f941f59e640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f942006dd10 0x7f9420137b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 -- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f942015a150 msgr2=0x7f94201393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f942015a150 0x7f94201393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 -- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f942006dd10 msgr2=0x7f9420137b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f942006dd10 0x7f9420137b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f9417fff640 1 -- 172.21.15.116:0/2916459308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942006e820 con 0x7f94201398f0 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f941f59e640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f942006dd10 0x7f9420137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:34.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.014+0000 7f941ed9d640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f942015a150 0x7f94201393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.015+0000 7f9417fff640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 0x7f942006d7d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f940800b7f0 tx=0x7f940800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.015+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408010830 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.015+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9408010990 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.015+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942006eb00 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.015+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408010d20 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.015+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942015a530 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.016+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94200cbe40 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.016+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f9408014430 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.017+0000 7f94177fe640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f940006bc10 0x7f940006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.017+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f940808b250 con 0x7f94201398f0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.017+0000 7f941ed9d640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f940006bc10 0x7f940006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.017+0000 7f941ed9d640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f940006bc10 0x7f940006e0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9410006fd0 tx=0x7f9410008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.013 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.021+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9408055610 con 0x7f94201398f0 2024-02-18T11:48:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.192+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9420074520 con 0x7f94201398f0 2024-02-18T11:48:34.184 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.192+0000 7f94177fe640 1 -- 172.21.15.116:0/2916459308 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9408058cd0 con 0x7f94201398f0 2024-02-18T11:48:34.184 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T11:48:34.186 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f940006bc10 msgr2=0x7f940006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f940006bc10 0x7f940006e0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9410006fd0 tx=0x7f9410008040 comp rx=0 tx=0).stop 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 msgr2=0x7f942006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 0x7f942006d7d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f940800b7f0 tx=0x7f940800bcb0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 shutdown_connections 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f940006bc10 0x7f940006e0d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f942006dd10 0x7f9420137b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94201398f0 0x7f942006d7d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 --2- 172.21.15.116:0/2916459308 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f942015a150 0x7f94201393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 >> 172.21.15.116:0/2916459308 conn(0x7f9420067000 msgr2=0x7f94200711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.187 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 shutdown_connections 2024-02-18T11:48:34.188 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.194+0000 7f942505c640 1 -- 172.21.15.116:0/2916459308 wait complete. 2024-02-18T11:48:34.197 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:34.293 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:48:34.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:48:34.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.305+0000 7fe9dd7a0640 1 Processor -- start 2024-02-18T11:48:34.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.305+0000 7fe9dd7a0640 1 -- start start 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dd7a0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dd7a0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe9d80b3ce0 0x7fe9d815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dd7a0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d80acd90 con 0x7fe9d8143140 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dd7a0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d8143a50 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dd7a0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80b3850 con 0x7fe9d806ef60 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dcf9f640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe9d8143140 0x7fe9d806d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55654/0 (socket says 172.21.15.116:55654) 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9dcf9f640 1 -- 172.21.15.116:0/1004462655 learned_addr learned my addr 172.21.15.116:0/1004462655 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9d77fe640 1 --2- 172.21.15.116:0/1004462655 >> v2:172.21.15.143:3300/0 conn(0x7fe9d80b3ce0 0x7fe9d815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9d7fff640 1 --2- 172.21.15.116:0/1004462655 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9d6ffd640 1 -- 172.21.15.116:0/1004462655 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556199287 0 0) 0x7fe9d80acd90 con 0x7fe9d8143140 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.306+0000 7fe9d6ffd640 1 -- 172.21.15.116:0/1004462655 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c0003520 con 0x7fe9d8143140 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.307+0000 7fe9d7fff640 1 -- 172.21.15.116:0/1004462655 >> v2:172.21.15.143:3300/0 conn(0x7fe9d80b3ce0 msgr2=0x7fe9d815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.307+0000 7fe9d7fff640 1 --2- 172.21.15.116:0/1004462655 >> v2:172.21.15.143:3300/0 conn(0x7fe9d80b3ce0 0x7fe9d815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.307+0000 7fe9d7fff640 1 -- 172.21.15.116:0/1004462655 >> v1:172.21.15.116:6789/0 conn(0x7fe9d8143140 legacy=0x7fe9d806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.307+0000 7fe9d7fff640 1 -- 172.21.15.116:0/1004462655 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d80ad2c0 con 0x7fe9d806ef60 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.307+0000 7fe9d77fe640 1 --2- 172.21.15.116:0/1004462655 >> v2:172.21.15.143:3300/0 conn(0x7fe9d80b3ce0 0x7fe9d815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.307+0000 7fe9d7fff640 1 --2- 172.21.15.116:0/1004462655 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d815d7d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0097a0 tx=0x7fe9cc02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4907800df2e9dd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9d6ffd640 1 -- 172.21.15.116:0/1004462655 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc030610 con 0x7fe9d806ef60 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9d6ffd640 1 -- 172.21.15.116:0/1004462655 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc030770 con 0x7fe9d806ef60 2024-02-18T11:48:34.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9d6ffd640 1 -- 172.21.15.116:0/1004462655 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc030b00 con 0x7fe9d806ef60 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/1004462655 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 msgr2=0x7fe9d815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/1004462655 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d815d7d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0097a0 tx=0x7fe9cc02b960 comp rx=0 tx=0).stop 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/1004462655 shutdown_connections 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/1004462655 >> v2:172.21.15.143:3300/0 conn(0x7fe9d80b3ce0 0x7fe9d815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/1004462655 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d815d7d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/1004462655 >> 172.21.15.116:0/1004462655 conn(0x7fe9d8072640 msgr2=0x7fe9d806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/1004462655 shutdown_connections 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.308+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/1004462655 wait complete. 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.309+0000 7fe9dd7a0640 1 Processor -- start 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 -- start start 2024-02-18T11:48:34.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d81595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 0x7fe9d80ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9d8143140 0x7fe9d809a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8099f80 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d8159b20 con 0x7fe9d8143140 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9dd7a0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80cf5c0 con 0x7fe9d806ef60 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 0x7fe9d80ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.310+0000 7fe9d77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 0x7fe9d80ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35264/0 (socket says 172.21.15.116:35264) 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 -- 172.21.15.116:0/3334919398 learned_addr learned my addr 172.21.15.116:0/3334919398 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d7fff640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d81595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9dcf9f640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9d8143140 0x7fe9d809a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 -- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 msgr2=0x7fe9d81595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d81595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 -- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9d8143140 msgr2=0x7fe9d809a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9d8143140 0x7fe9d809a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 -- 172.21.15.116:0/3334919398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c8009400 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d77fe640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 0x7fe9d80ced30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800ec10 tx=0x7fe9c800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8009760 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c800cc30 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc009400 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8013470 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.311+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d8099c10 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.312+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d80cfae0 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.313+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe9c80135d0 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.313+0000 7fe9d4ff9640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe9c406bc70 0x7fe9c406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.313+0000 7fe9d7fff640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe9c406bc70 0x7fe9c406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.314+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe9c808ade0 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.314+0000 7fe9d7fff640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe9c406bc70 0x7fe9c406e130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc036000 tx=0x7fe9cc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.318+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9c8055250 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.527+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d8099a50 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.527+0000 7fe9d4ff9640 1 -- 172.21.15.116:0/3334919398 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fe9c8058910 con 0x7fe9d80b3ce0 2024-02-18T11:48:34.520 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:34.520 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:34.520 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe9c406bc70 msgr2=0x7fe9c406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe9c406bc70 0x7fe9c406e130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc036000 tx=0x7fe9cc008030 comp rx=0 tx=0).stop 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 msgr2=0x7fe9d80ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 0x7fe9d80ced30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800ec10 tx=0x7fe9c800c520 comp rx=0 tx=0).stop 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 shutdown_connections 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe9c406bc70 0x7fe9c406e130 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9d8143140 0x7fe9d809a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d80b3ce0 0x7fe9d80ced30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 --2- 172.21.15.116:0/3334919398 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe9d806ef60 0x7fe9d81595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 >> 172.21.15.116:0/3334919398 conn(0x7fe9d8072640 msgr2=0x7fe9d8142260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 shutdown_connections 2024-02-18T11:48:34.523 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.529+0000 7fe9dd7a0640 1 -- 172.21.15.116:0/3334919398 wait complete. 2024-02-18T11:48:34.532 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 Processor -- start 2024-02-18T11:48:34.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 -- start start 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4fd415dc60 0x7f4fd40b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4fd40d1080 0x7f4fd40ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40ce780 con 0x7f4fd4161850 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd40d3730 con 0x7f4fd40d1080 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fdb1a7640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd40cf250 con 0x7f4fd415dc60 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.642+0000 7f4fd9744640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4fd40d1080 0x7f4fd40ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd3fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4fd4161850 0x7f4fd40d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55670/0 (socket says 172.21.15.116:55670) 2024-02-18T11:48:34.637 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd3fff640 1 -- 172.21.15.116:0/230147269 learned_addr learned my addr 172.21.15.116:0/230147269 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd8f43640 1 --2- 172.21.15.116:0/230147269 >> v2:172.21.15.143:3300/0 conn(0x7f4fd415dc60 0x7f4fd40b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd9744640 1 -- 172.21.15.116:0/230147269 >> v2:172.21.15.143:3300/0 conn(0x7f4fd415dc60 msgr2=0x7f4fd40b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd9744640 1 --2- 172.21.15.116:0/230147269 >> v2:172.21.15.143:3300/0 conn(0x7f4fd415dc60 0x7f4fd40b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd9744640 1 -- 172.21.15.116:0/230147269 >> v1:172.21.15.116:6789/0 conn(0x7f4fd4161850 legacy=0x7f4fd40d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd9744640 1 -- 172.21.15.116:0/230147269 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40ce9b0 con 0x7f4fd40d1080 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.643+0000 7f4fd37fe640 1 -- 172.21.15.116:0/230147269 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812184228 0 0) 0x7f4fd40ce780 con 0x7f4fd4161850 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fd9744640 1 --2- 172.21.15.116:0/230147269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40d1080 0x7f4fd40ce240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4fd408c680 tx=0x7f4fcc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb6b3318a83e8fa server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fd37fe640 1 -- 172.21.15.116:0/230147269 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc02cc80 con 0x7f4fd40d1080 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fd37fe640 1 -- 172.21.15.116:0/230147269 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc02cde0 con 0x7f4fd40d1080 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fd37fe640 1 -- 172.21.15.116:0/230147269 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc034660 con 0x7f4fd40d1080 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/230147269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40d1080 msgr2=0x7f4fd40ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.638 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/230147269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40d1080 0x7f4fd40ce240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4fd408c680 tx=0x7f4fcc02bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/230147269 shutdown_connections 2024-02-18T11:48:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/230147269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd40d1080 0x7f4fd40ce240 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/230147269 >> v2:172.21.15.143:3300/0 conn(0x7f4fd415dc60 0x7f4fd40b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/230147269 >> 172.21.15.116:0/230147269 conn(0x7f4fd40b5910 msgr2=0x7f4fd40b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/230147269 shutdown_connections 2024-02-18T11:48:34.639 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.644+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/230147269 wait complete. 2024-02-18T11:48:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.645+0000 7f4fdb1a7640 1 Processor -- start 2024-02-18T11:48:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.645+0000 7f4fdb1a7640 1 -- start start 2024-02-18T11:48:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fdb1a7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fd40d1080 0x7f4fd4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fdb1a7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 0x7f4fd40d3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fdb1a7640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4fd4161850 0x7f4fd40d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fdb1a7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd415ecb0 con 0x7f4fd415dc60 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fdb1a7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd415e380 con 0x7f4fd40d1080 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fdb1a7640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd415e660 con 0x7f4fd4161850 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fd3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 0x7f4fd40d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fd3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 0x7f4fd40d3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35272/0 (socket says 172.21.15.116:35272) 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fd3fff640 1 -- 172.21.15.116:0/1694361941 learned_addr learned my addr 172.21.15.116:0/1694361941 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fd8f43640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fd40d1080 0x7f4fd4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.646+0000 7f4fd9744640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4fd4161850 0x7f4fd40d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd3fff640 1 -- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4fd4161850 msgr2=0x7f4fd40d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd3fff640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4fd4161850 0x7f4fd40d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd3fff640 1 -- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fd40d1080 msgr2=0x7f4fd4153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd3fff640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fd40d1080 0x7f4fd4153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd3fff640 1 -- 172.21.15.116:0/1694361941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc009a70 con 0x7f4fd415dc60 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd8f43640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fd40d1080 0x7f4fd4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd3fff640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 0x7f4fd40d3970 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c430 tx=0x7f4fc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc00074e0 con 0x7f4fd415dc60 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc0007660 con 0x7f4fd415dc60 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0004a20 con 0x7f4fd415dc60 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd415e940 con 0x7f4fd415dc60 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.647+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41f4370 con 0x7f4fd415dc60 2024-02-18T11:48:34.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.648+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd4067840 con 0x7f4fd415dc60 2024-02-18T11:48:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.648+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4fc0004b80 con 0x7f4fd415dc60 2024-02-18T11:48:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.649+0000 7f4fd17fa640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4fb806bc70 0x7f4fb806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.649+0000 7f4fd8f43640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4fb806bc70 0x7f4fb806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.649+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f4fc008aa40 con 0x7f4fd415dc60 2024-02-18T11:48:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.649+0000 7f4fd8f43640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4fb806bc70 0x7f4fb806e130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006fd0 tx=0x7f4fc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.653+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fc008ce60 con 0x7f4fd415dc60 2024-02-18T11:48:34.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.827+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fd415ead0 con 0x7f4fd415dc60 2024-02-18T11:48:34.820 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.828+0000 7f4fd17fa640 1 -- 172.21.15.116:0/1694361941 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f4fc0054e00 con 0x7f4fd415dc60 2024-02-18T11:48:34.820 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:34.821 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:34.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4fb806bc70 msgr2=0x7f4fb806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.822 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4fb806bc70 0x7f4fb806e130 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006fd0 tx=0x7f4fc8008040 comp rx=0 tx=0).stop 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 msgr2=0x7f4fd40d3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 0x7f4fd40d3970 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c430 tx=0x7f4fc000c8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 shutdown_connections 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4fb806bc70 0x7f4fb806e130 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4fd4161850 0x7f4fd40d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fd415dc60 0x7f4fd40d3970 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 --2- 172.21.15.116:0/1694361941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fd40d1080 0x7f4fd4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 >> 172.21.15.116:0/1694361941 conn(0x7f4fd40b5910 msgr2=0x7f4fd406ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 shutdown_connections 2024-02-18T11:48:34.823 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.830+0000 7f4fdb1a7640 1 -- 172.21.15.116:0/1694361941 wait complete. 2024-02-18T11:48:34.833 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-18T11:48:34.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:34.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.941+0000 7fd84f50c640 1 Processor -- start 2024-02-18T11:48:34.934 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84f50c640 1 -- start start 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84f50c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd84815a860 0x7fd8480cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84f50c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8480cd1f0 0x7fd84806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84f50c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8480b02e0 con 0x7fd84815ac40 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84f50c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480b1780 con 0x7fd8480cd1f0 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84f50c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8480af6d0 con 0x7fd84815a860 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84caa7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8480cd1f0 0x7fd84806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84daa9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd84815ac40 0x7fd84815e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55678/0 (socket says 172.21.15.116:55678) 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84caa7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8480cd1f0 0x7fd84806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35286/0 (socket says 172.21.15.116:35286) 2024-02-18T11:48:34.936 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.942+0000 7fd84daa9640 1 -- 172.21.15.116:0/2158251403 learned_addr learned my addr 172.21.15.116:0/2158251403 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.943+0000 7fd84caa7640 1 -- 172.21.15.116:0/2158251403 >> v2:172.21.15.143:3300/0 conn(0x7fd84815a860 msgr2=0x7fd8480cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.943+0000 7fd84caa7640 1 --2- 172.21.15.116:0/2158251403 >> v2:172.21.15.143:3300/0 conn(0x7fd84815a860 0x7fd8480cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.943+0000 7fd84caa7640 1 -- 172.21.15.116:0/2158251403 >> v1:172.21.15.116:6789/0 conn(0x7fd84815ac40 legacy=0x7fd84815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.943+0000 7fd83ffff640 1 -- 172.21.15.116:0/2158251403 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306567250 0 0) 0x7fd8480b02e0 con 0x7fd84815ac40 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.943+0000 7fd84caa7640 1 -- 172.21.15.116:0/2158251403 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480af9b0 con 0x7fd8480cd1f0 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.943+0000 7fd84caa7640 1 --2- 172.21.15.116:0/2158251403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8480cd1f0 0x7fd84806e310 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd84806ef20 tx=0x7fd83802e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4aebd0a4559318 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd83ffff640 1 -- 172.21.15.116:0/2158251403 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83802ee50 con 0x7fd8480cd1f0 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd83ffff640 1 -- 172.21.15.116:0/2158251403 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838036440 con 0x7fd8480cd1f0 2024-02-18T11:48:34.937 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd83ffff640 1 -- 172.21.15.116:0/2158251403 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8380367d0 con 0x7fd8480cd1f0 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 -- 172.21.15.116:0/2158251403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8480cd1f0 msgr2=0x7fd84806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 --2- 172.21.15.116:0/2158251403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8480cd1f0 0x7fd84806e310 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd84806ef20 tx=0x7fd83802e0a0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 -- 172.21.15.116:0/2158251403 shutdown_connections 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 --2- 172.21.15.116:0/2158251403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8480cd1f0 0x7fd84806e310 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 --2- 172.21.15.116:0/2158251403 >> v2:172.21.15.143:3300/0 conn(0x7fd84815a860 0x7fd8480cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 -- 172.21.15.116:0/2158251403 >> 172.21.15.116:0/2158251403 conn(0x7fd848066f20 msgr2=0x7fd84806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 -- 172.21.15.116:0/2158251403 shutdown_connections 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.944+0000 7fd84f50c640 1 -- 172.21.15.116:0/2158251403 wait complete. 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.945+0000 7fd84f50c640 1 Processor -- start 2024-02-18T11:48:34.942 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.945+0000 7fd84f50c640 1 -- start start 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84f50c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8480cd1f0 0x7fd84814df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84f50c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd84815a860 0x7fd8480ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84f50c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 0x7fd848140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84f50c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84814e7c0 con 0x7fd84815ac40 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84f50c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848140c00 con 0x7fd8480cd1f0 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84f50c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481413d0 con 0x7fd84815a860 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84daa9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 0x7fd848140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84daa9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 0x7fd848140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35296/0 (socket says 172.21.15.116:35296) 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84daa9640 1 -- 172.21.15.116:0/446817420 learned_addr learned my addr 172.21.15.116:0/446817420 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84d2a8640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8480cd1f0 0x7fd84814df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.943 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.946+0000 7fd84caa7640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd84815a860 0x7fd8480ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84daa9640 1 -- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd84815a860 msgr2=0x7fd8480ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84daa9640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd84815a860 0x7fd8480ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84daa9640 1 -- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8480cd1f0 msgr2=0x7fd84814df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84daa9640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8480cd1f0 0x7fd84814df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84daa9640 1 -- 172.21.15.116:0/446817420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83800b6a0 con 0x7fd84815ac40 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84daa9640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 0x7fd848140e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c440 tx=0x7fd84400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8440074e0 con 0x7fd84815ac40 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd844007640 con 0x7fd84815ac40 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd848142210 con 0x7fd84815ac40 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844004a20 con 0x7fd84815ac40 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.947+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8481418e0 con 0x7fd84815ac40 2024-02-18T11:48:34.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.948+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8481423a0 con 0x7fd84815ac40 2024-02-18T11:48:34.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.948+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd844004b80 con 0x7fd84815ac40 2024-02-18T11:48:34.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.949+0000 7fd83dffb640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82c06bc70 0x7fd82c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:34.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.949+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd84408a6a0 con 0x7fd84815ac40 2024-02-18T11:48:34.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.949+0000 7fd84d2a8640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82c06bc70 0x7fd82c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:34.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.949+0000 7fd84d2a8640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82c06bc70 0x7fd82c06e130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd848156570 tx=0x7fd84000d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:34.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:34.953+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd844054c60 con 0x7fd84815ac40 2024-02-18T11:48:35.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.163+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd848074520 con 0x7fd84815ac40 2024-02-18T11:48:35.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.164+0000 7fd83dffb640 1 -- 172.21.15.116:0/446817420 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7fd844058320 con 0x7fd84815ac40 2024-02-18T11:48:35.157 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:35.157 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:35.159 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82c06bc70 msgr2=0x7fd82c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82c06bc70 0x7fd82c06e130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd848156570 tx=0x7fd84000d040 comp rx=0 tx=0).stop 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 msgr2=0x7fd848140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 0x7fd848140e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c440 tx=0x7fd84400c900 comp rx=0 tx=0).stop 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 shutdown_connections 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82c06bc70 0x7fd82c06e130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84815ac40 0x7fd848140e90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd84815a860 0x7fd8480ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 --2- 172.21.15.116:0/446817420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8480cd1f0 0x7fd84814df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 >> 172.21.15.116:0/446817420 conn(0x7fd848066f20 msgr2=0x7fd8480b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 shutdown_connections 2024-02-18T11:48:35.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.166+0000 7fd84f50c640 1 -- 172.21.15.116:0/446817420 wait complete. 2024-02-18T11:48:35.169 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:35.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.278+0000 7f186c0c0640 1 Processor -- start 2024-02-18T11:48:35.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.278+0000 7f186c0c0640 1 -- start start 2024-02-18T11:48:35.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186c0c0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f186415ac40 0x7f186406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186c0c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f186406e850 0x7f18640b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186c0c0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186406eca0 con 0x7f186415a860 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186c0c0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18640b02e0 con 0x7f186406e850 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186c0c0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18640af6d0 con 0x7f186415ac40 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186a65d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f186406e850 0x7f18640b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f1869e5c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f186415a860 0x7f18640cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55680/0 (socket says 172.21.15.116:55680) 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f1869e5c640 1 -- 172.21.15.116:0/2551612457 learned_addr learned my addr 172.21.15.116:0/2551612457 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186965b640 1 --2- 172.21.15.116:0/2551612457 >> v2:172.21.15.143:3300/0 conn(0x7f186415ac40 0x7f186406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186a65d640 1 -- 172.21.15.116:0/2551612457 >> v2:172.21.15.143:3300/0 conn(0x7f186415ac40 msgr2=0x7f186406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186a65d640 1 --2- 172.21.15.116:0/2551612457 >> v2:172.21.15.143:3300/0 conn(0x7f186415ac40 0x7f186406e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186a65d640 1 -- 172.21.15.116:0/2551612457 >> v1:172.21.15.116:6789/0 conn(0x7f186415a860 legacy=0x7f18640cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f186a65d640 1 -- 172.21.15.116:0/2551612457 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18640af9b0 con 0x7f186406e850 2024-02-18T11:48:35.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.279+0000 7f1868e5a640 1 -- 172.21.15.116:0/2551612457 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392935632 0 0) 0x7f1860001110 con 0x7f186415a860 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.280+0000 7f186a65d640 1 --2- 172.21.15.116:0/2551612457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186406e850 0x7f18640b0770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f18640b2610 tx=0x7f185c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a67fe2e9957de6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.280+0000 7f1868e5a640 1 -- 172.21.15.116:0/2551612457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c02cc80 con 0x7f186406e850 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.280+0000 7f1868e5a640 1 -- 172.21.15.116:0/2551612457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c02cde0 con 0x7f186406e850 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.280+0000 7f1868e5a640 1 -- 172.21.15.116:0/2551612457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c034660 con 0x7f186406e850 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 -- 172.21.15.116:0/2551612457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186406e850 msgr2=0x7f18640b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 --2- 172.21.15.116:0/2551612457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186406e850 0x7f18640b0770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f18640b2610 tx=0x7f185c02bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 -- 172.21.15.116:0/2551612457 shutdown_connections 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 --2- 172.21.15.116:0/2551612457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186406e850 0x7f18640b0770 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 --2- 172.21.15.116:0/2551612457 >> v2:172.21.15.143:3300/0 conn(0x7f186415ac40 0x7f186406e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 -- 172.21.15.116:0/2551612457 >> 172.21.15.116:0/2551612457 conn(0x7f1864066f20 msgr2=0x7f186406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 -- 172.21.15.116:0/2551612457 shutdown_connections 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.281+0000 7f186c0c0640 1 -- 172.21.15.116:0/2551612457 wait complete. 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.282+0000 7f186c0c0640 1 Processor -- start 2024-02-18T11:48:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.282+0000 7f186c0c0640 1 -- start start 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.282+0000 7f186c0c0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f186406e850 0x7f18641e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.282+0000 7f186c0c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186415a860 0x7f18641f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.282+0000 7f186c0c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 0x7f1864143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186c0c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641527d0 con 0x7f186415ac40 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186c0c0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1864152ab0 con 0x7f186415a860 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186c0c0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1864152d90 con 0x7f186406e850 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 0x7f1864143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 0x7f1864143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35324/0 (socket says 172.21.15.116:35324) 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 -- 172.21.15.116:0/1446497215 learned_addr learned my addr 172.21.15.116:0/1446497215 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:35.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186965b640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186415a860 0x7f18641f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f1869e5c640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f186406e850 0x7f18641e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 -- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f186406e850 msgr2=0x7f18641e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f186406e850 0x7f18641e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 -- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186415a860 msgr2=0x7f18641f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186415a860 0x7f18641f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 -- 172.21.15.116:0/1446497215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009a70 con 0x7f186415ac40 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186965b640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186415a860 0x7f18641f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f1869e5c640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f186406e850 0x7f18641e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186a65d640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 0x7f1864143070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f185c009e80 tx=0x7f185c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c009450 con 0x7f186415ac40 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c040450 con 0x7f186415ac40 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1864153070 con 0x7f186415ac40 2024-02-18T11:48:35.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c040800 con 0x7f186415ac40 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.283+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641e4e40 con 0x7f186415ac40 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.284+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1864067510 con 0x7f186415ac40 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.285+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f185c040960 con 0x7f186415ac40 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.285+0000 7f185a7fc640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f184806bca0 0x7f184806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.285+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f185c0ad550 con 0x7f186415ac40 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.285+0000 7f1869e5c640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f184806bca0 0x7f184806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.285+0000 7f1869e5c640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f184806bca0 0x7f184806e160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f60 tx=0x7f1860007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.289+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f185c077a60 con 0x7f186415ac40 2024-02-18T11:48:35.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.494+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1864074520 con 0x7f186415ac40 2024-02-18T11:48:35.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.494+0000 7f185a7fc640 1 -- 172.21.15.116:0/1446497215 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4940 (secure 0 0 0) 0x7f185c07b120 con 0x7f186415ac40 2024-02-18T11:48:35.487 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:35.487 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/4284420195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":4284420195},{"type":"v1","addr":"172.21.15.143:6835","nonce":4284420195}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/757546214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":757546214},{"type":"v1","addr":"172.21.15.143:6834","nonce":757546214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/869975983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":869975983},{"type":"v1","addr":"172.21.15.116:6837","nonce":869975983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:47:33.859710+0000","modified":"2024-02-18T11:47:55.942466+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1897601775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1897601775},{"type":"v1","addr":"172.21.15.116:6836","nonce":1897601775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-18T11:48:35.487 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 8 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f184806bca0 msgr2=0x7f184806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f184806bca0 0x7f184806e160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f60 tx=0x7f1860007a50 comp rx=0 tx=0).stop 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 msgr2=0x7f1864143070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 0x7f1864143070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f185c009e80 tx=0x7f185c004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 shutdown_connections 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f184806bca0 0x7f184806e160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186415ac40 0x7f1864143070 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f186415a860 0x7f18641f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 --2- 172.21.15.116:0/1446497215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f186406e850 0x7f18641e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 >> 172.21.15.116:0/1446497215 conn(0x7f1864066f20 msgr2=0x7f1864156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:35.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 shutdown_connections 2024-02-18T11:48:35.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.496+0000 7f186c0c0640 1 -- 172.21.15.116:0/1446497215 wait complete. 2024-02-18T11:48:35.499 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:35.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.611+0000 7f74866c3640 1 Processor -- start 2024-02-18T11:48:35.603 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.611+0000 7f74866c3640 1 -- start start 2024-02-18T11:48:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f74866c3640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f748015efb0 0x7f74800d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f74866c3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7480099a50 0x7f7480099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f74866c3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800d0a80 con 0x7f74800cf900 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f74866c3640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748015f390 con 0x7f748015efb0 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f74866c3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748009a400 con 0x7f7480099a50 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747f7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f74800cf900 0x7f748006c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55688/0 (socket says 172.21.15.116:55688) 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747f7fe640 1 -- 172.21.15.116:0/929359384 learned_addr learned my addr 172.21.15.116:0/929359384 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f7484c60640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7480099a50 0x7f7480099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747ffff640 1 --2- 172.21.15.116:0/929359384 >> v2:172.21.15.143:3300/0 conn(0x7f748015efb0 0x7f74800d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747effd640 1 -- 172.21.15.116:0/929359384 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554127840 0 0) 0x7f74800d0a80 con 0x7f74800cf900 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747effd640 1 -- 172.21.15.116:0/929359384 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7468003520 con 0x7f74800cf900 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747ffff640 1 -- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7480099a50 msgr2=0x7f7480099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747ffff640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7480099a50 0x7f7480099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747ffff640 1 -- 172.21.15.116:0/929359384 >> v1:172.21.15.116:6789/0 conn(0x7f74800cf900 legacy=0x7f748006c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f747ffff640 1 -- 172.21.15.116:0/929359384 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800cfe70 con 0x7f748015efb0 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.612+0000 7f7484c60640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7480099a50 0x7f7480099ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.613+0000 7f747ffff640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748015efb0 0x7f74800d0f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74700097a0 tx=0x7f747002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc4571a4957301a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.613+0000 7f747effd640 1 -- 172.21.15.116:0/929359384 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747002e070 con 0x7f748015efb0 2024-02-18T11:48:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.613+0000 7f747effd640 1 -- 172.21.15.116:0/929359384 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74700315a0 con 0x7f748015efb0 2024-02-18T11:48:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.613+0000 7f747effd640 1 -- 172.21.15.116:0/929359384 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470031910 con 0x7f748015efb0 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 -- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748015efb0 msgr2=0x7f74800d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748015efb0 0x7f74800d0f10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f74700097a0 tx=0x7f747002b840 comp rx=0 tx=0).stop 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 -- 172.21.15.116:0/929359384 shutdown_connections 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7480099a50 0x7f7480099ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 --2- 172.21.15.116:0/929359384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748015efb0 0x7f74800d0f10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 -- 172.21.15.116:0/929359384 >> 172.21.15.116:0/929359384 conn(0x7f74800d1790 msgr2=0x7f74800d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 -- 172.21.15.116:0/929359384 shutdown_connections 2024-02-18T11:48:35.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.614+0000 7f74866c3640 1 -- 172.21.15.116:0/929359384 wait complete. 2024-02-18T11:48:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.616+0000 7f74866c3640 1 Processor -- start 2024-02-18T11:48:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.616+0000 7f74866c3640 1 -- start start 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f74866c3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7480099a50 0x7f748014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f74866c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 0x7f74800c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f74866c3640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748015efb0 0x7f74800c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f74866c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748014c3f0 con 0x7f74800cf900 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f74866c3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480160420 con 0x7f7480099a50 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f74866c3640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7480160700 con 0x7f748015efb0 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.617+0000 7f747f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 0x7f74800c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 0x7f74800c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35338/0 (socket says 172.21.15.116:35338) 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 -- 172.21.15.116:0/2060356855 learned_addr learned my addr 172.21.15.116:0/2060356855 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f7484c60640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748015efb0 0x7f74800c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747ffff640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7480099a50 0x7f748014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 -- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748015efb0 msgr2=0x7f74800c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748015efb0 0x7f74800c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 -- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7480099a50 msgr2=0x7f748014ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7480099a50 0x7f748014ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 -- 172.21.15.116:0/2060356855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478009a70 con 0x7f74800cf900 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747f7fe640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 0x7f74800c5bd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f747400c430 tx=0x7f747400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474005ce0 con 0x7f74800cf900 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474005e40 con 0x7f74800cf900 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7470009400 con 0x7f74800cf900 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740072f0 con 0x7f74800cf900 2024-02-18T11:48:35.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.618+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7480160cc0 con 0x7f74800cf900 2024-02-18T11:48:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.619+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7480067840 con 0x7f74800cf900 2024-02-18T11:48:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.620+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7474007450 con 0x7f74800cf900 2024-02-18T11:48:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.620+0000 7f747cff9640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746c06bc70 0x7f746c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.620+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f747408a130 con 0x7f74800cf900 2024-02-18T11:48:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.620+0000 7f747ffff640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746c06bc70 0x7f746c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.621+0000 7f747ffff640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746c06bc70 0x7f746c06e130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7470036000 tx=0x7f747003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.620 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.628+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74740545a0 con 0x7f74800cf900 2024-02-18T11:48:35.796 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.803+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7480074520 con 0x7f74800cf900 2024-02-18T11:48:35.797 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.804+0000 7f747cff9640 1 -- 172.21.15.116:0/2060356855 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18963 (secure 0 0 0) 0x7f7474057c60 con 0x7f74800cf900 2024-02-18T11:48:35.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:35.797 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":26,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:15.686964+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:47:32.328152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-18T11:47:32.824865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-18T11:47:36.561926+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:35.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746c06bc70 msgr2=0x7f746c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746c06bc70 0x7f746c06e130 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7470036000 tx=0x7f747003d040 comp rx=0 tx=0).stop 2024-02-18T11:48:35.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 msgr2=0x7f74800c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 0x7f74800c5bd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f747400c430 tx=0x7f747400c830 comp rx=0 tx=0).stop 2024-02-18T11:48:35.799 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 shutdown_connections 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746c06bc70 0x7f746c06e130 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748015efb0 0x7f74800c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74800cf900 0x7f74800c5bd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 --2- 172.21.15.116:0/2060356855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7480099a50 0x7f748014ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 >> 172.21.15.116:0/2060356855 conn(0x7f74800d1790 msgr2=0x7f74801597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 shutdown_connections 2024-02-18T11:48:35.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.807+0000 7f74866c3640 1 -- 172.21.15.116:0/2060356855 wait complete. 2024-02-18T11:48:35.810 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-18T11:48:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.920+0000 7f5130ed4640 1 Processor -- start 2024-02-18T11:48:35.912 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.920+0000 7f5130ed4640 1 -- start start 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.920+0000 7f5130ed4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f512c1617e0 0x7f512c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.920+0000 7f5130ed4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0cf0d0 con 0x7f512c0cec80 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.920+0000 7f5130ed4640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512c0d1ee0 con 0x7f512c15dbf0 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.920+0000 7f5130ed4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c161dc0 con 0x7f512c1617e0 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f512ad9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f512c0cec80 0x7f512c0d2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55700/0 (socket says 172.21.15.116:55700) 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f512ad9d640 1 -- 172.21.15.116:0/2272617206 learned_addr learned my addr 172.21.15.116:0/2272617206 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:35.914 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5123fff640 1 --2- 172.21.15.116:0/2272617206 >> v2:172.21.15.143:3300/0 conn(0x7f512c1617e0 0x7f512c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898424709 0 0) 0x7f5124001490 con 0x7f512c0cec80 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5118003520 con 0x7f512c0cec80 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317021331 0 0) 0x7f512c0d1ee0 con 0x7f512c15dbf0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5124001490 con 0x7f512c15dbf0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3954871675 0 0) 0x7f5118003520 con 0x7f512c0cec80 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5123fff640 1 -- 172.21.15.116:0/2272617206 >> v1:172.21.15.143:6789/0 conn(0x7f512c15dbf0 legacy=0x7f512c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5123fff640 1 -- 172.21.15.116:0/2272617206 >> v1:172.21.15.116:6789/0 conn(0x7f512c0cec80 legacy=0x7f512c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.921+0000 7f5123fff640 1 -- 172.21.15.116:0/2272617206 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c1620a0 con 0x7f512c1617e0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.922+0000 7f5123fff640 1 --2- 172.21.15.116:0/2272617206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 0x7f512c0ce740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51140097a0 tx=0x7f511402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25bbe3858dbdbf18 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.922+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124001b80 con 0x7f512c1617e0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.922+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511402e050 con 0x7f512c1617e0 2024-02-18T11:48:35.915 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.922+0000 7f5129d9b640 1 -- 172.21.15.116:0/2272617206 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114030730 con 0x7f512c1617e0 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.922+0000 7f5130ed4640 1 -- 172.21.15.116:0/2272617206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 msgr2=0x7f512c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.923+0000 7f5130ed4640 1 --2- 172.21.15.116:0/2272617206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 0x7f512c0ce740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f51140097a0 tx=0x7f511402b790 comp rx=0 tx=0).stop 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.923+0000 7f5130ed4640 1 -- 172.21.15.116:0/2272617206 shutdown_connections 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.923+0000 7f5130ed4640 1 --2- 172.21.15.116:0/2272617206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 0x7f512c0ce740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.923+0000 7f5130ed4640 1 -- 172.21.15.116:0/2272617206 >> 172.21.15.116:0/2272617206 conn(0x7f512c0b5480 msgr2=0x7f512c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.923+0000 7f5130ed4640 1 -- 172.21.15.116:0/2272617206 shutdown_connections 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.923+0000 7f5130ed4640 1 -- 172.21.15.116:0/2272617206 wait complete. 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.924+0000 7f5130ed4640 1 Processor -- start 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.924+0000 7f5130ed4640 1 -- start start 2024-02-18T11:48:35.918 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 0x7f512c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f512c15dbf0 0x7f512c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 0x7f512c0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c15f590 con 0x7f512c0cec80 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c15fdf0 con 0x7f512c1617e0 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c152fb0 con 0x7f512c15dbf0 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 0x7f512c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 0x7f512c1554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35342/0 (socket says 172.21.15.116:35342) 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 -- 172.21.15.116:0/1930137335 learned_addr learned my addr 172.21.15.116:0/1930137335 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 -- 172.21.15.116:0/1930137335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f512c15dbf0 msgr2=0x7f512c0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f512c15dbf0 0x7f512c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 -- 172.21.15.116:0/1930137335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 msgr2=0x7f512c0cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 0x7f512c0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:35.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 -- 172.21.15.116:0/1930137335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114009400 con 0x7f512c0cec80 2024-02-18T11:48:35.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f512a59c640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 0x7f512c1554d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c490 tx=0x7f511c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0052d0 con 0x7f512c0cec80 2024-02-18T11:48:35.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c005430 con 0x7f512c0cec80 2024-02-18T11:48:35.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c15e150 con 0x7f512c0cec80 2024-02-18T11:48:35.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c012870 con 0x7f512c0cec80 2024-02-18T11:48:35.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.925+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c15fa80 con 0x7f512c0cec80 2024-02-18T11:48:35.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.926+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c067840 con 0x7f512c0cec80 2024-02-18T11:48:35.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.927+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f511c0129d0 con 0x7f512c0cec80 2024-02-18T11:48:35.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.927+0000 7f5122ffd640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f510c06bc70 0x7f510c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:35.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.927+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f511c08a7f0 con 0x7f512c0cec80 2024-02-18T11:48:35.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.927+0000 7f5123fff640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f510c06bc70 0x7f510c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:35.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.928+0000 7f5123fff640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f510c06bc70 0x7f510c06e130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5114034000 tx=0x7f511402d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:35.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:35.931+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f511c054bb0 con 0x7f512c0cec80 2024-02-18T11:48:36.129 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.135+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f512c074520 con 0x7f512c0cec80 2024-02-18T11:48:36.130 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-02-18T11:48:36.136+0000 7f8c00a5e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-18T11:48:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.139+0000 7f5122ffd640 1 -- 172.21.15.116:0/1930137335 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f511c058270 con 0x7f512c0cec80 2024-02-18T11:48:36.131 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-18T11:48:36.134 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f510c06bc70 msgr2=0x7f510c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f510c06bc70 0x7f510c06e130 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5114034000 tx=0x7f511402d000 comp rx=0 tx=0).stop 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 msgr2=0x7f512c1554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 0x7f512c1554d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f511c00c490 tx=0x7f511c00c950 comp rx=0 tx=0).stop 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 shutdown_connections 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f510c06bc70 0x7f510c06e130 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f512c1617e0 0x7f512c0cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f512c15dbf0 0x7f512c0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 --2- 172.21.15.116:0/1930137335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f512c0cec80 0x7f512c1554d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 >> 172.21.15.116:0/1930137335 conn(0x7f512c0b5480 msgr2=0x7f512c1583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 shutdown_connections 2024-02-18T11:48:36.135 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.142+0000 7f5130ed4640 1 -- 172.21.15.116:0/1930137335 wait complete. 2024-02-18T11:48:36.137 INFO:tasks.ceph.mds.a.smithi116.stderr: -4983> 2024-02-18T11:47:38.939+0000 7fb1126b5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-18T11:48:36.147 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-18T11:48:36.168 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-02-18T11:48:36.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.258+0000 7fa293676640 1 Processor -- start 2024-02-18T11:48:36.251 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.259+0000 7fa293676640 1 -- start start 2024-02-18T11:48:36.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.259+0000 7fa293676640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c06ebf0 0x7fa28c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:36.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa293676640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa28c0b2db0 0x7fa28c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa293676640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c06ab60 con 0x7fa28c15dd90 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa293676640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c0cf940 con 0x7fa28c0b2db0 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa293676640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c06adb0 con 0x7fa28c06ebf0 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa28c0b2db0 0x7fa28c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa290c11640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c06ebf0 0x7fa28c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa28c0b2db0 0x7fa28c0b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:53172/0 (socket says 172.21.15.116:53172) 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 -- 172.21.15.116:0/262583967 learned_addr learned my addr 172.21.15.116:0/262583967 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291412640 1 --1- 172.21.15.116:0/262583967 >> v1:172.21.15.143:6789/0 conn(0x7fa28c15dd90 0x7fa28c0ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:56230/0 (socket says 172.21.15.116:56230) 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 -- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c06ebf0 msgr2=0x7fa28c06f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 --2- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c06ebf0 0x7fa28c06f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 -- 172.21.15.116:0/262583967 >> v1:172.21.15.143:6789/0 conn(0x7fa28c15dd90 legacy=0x7fa28c0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa291c13640 1 -- 172.21.15.116:0/262583967 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c06b090 con 0x7fa28c0b2db0 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa283fff640 1 -- 172.21.15.116:0/262583967 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762644164 0 0) 0x7fa28c06ab60 con 0x7fa28c15dd90 2024-02-18T11:48:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.260+0000 7fa290c11640 1 --2- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c06ebf0 0x7fa28c06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.261+0000 7fa291c13640 1 --2- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c0b2db0 0x7fa28c0b3240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa284009e50 tx=0x7fa28402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37448a248c81580a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.261+0000 7fa283fff640 1 -- 172.21.15.116:0/262583967 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28402cb00 con 0x7fa28c0b2db0 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.261+0000 7fa283fff640 1 -- 172.21.15.116:0/262583967 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28402cc60 con 0x7fa28c0b2db0 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.261+0000 7fa283fff640 1 -- 172.21.15.116:0/262583967 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840343f0 con 0x7fa28c0b2db0 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 -- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c0b2db0 msgr2=0x7fa28c0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 --2- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c0b2db0 0x7fa28c0b3240 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa284009e50 tx=0x7fa28402bf60 comp rx=0 tx=0).stop 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 -- 172.21.15.116:0/262583967 shutdown_connections 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 --2- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c0b2db0 0x7fa28c0b3240 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.255 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 --2- 172.21.15.116:0/262583967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c06ebf0 0x7fa28c06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 -- 172.21.15.116:0/262583967 >> 172.21.15.116:0/262583967 conn(0x7fa28c0cfd80 msgr2=0x7fa28c0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:36.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 -- 172.21.15.116:0/262583967 shutdown_connections 2024-02-18T11:48:36.256 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.262+0000 7fa293676640 1 -- 172.21.15.116:0/262583967 wait complete. 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.263+0000 7fa293676640 1 Processor -- start 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.263+0000 7fa293676640 1 -- start start 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c06ebf0 0x7fa28c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c0b2db0 0x7fa28c155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 0x7fa28c155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c1f4000 con 0x7fa28c15dd90 2024-02-18T11:48:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c155fb0 con 0x7fa28c06ebf0 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c1f33a0 con 0x7fa28c0b2db0 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 0x7fa28c155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 0x7fa28c155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35358/0 (socket says 172.21.15.116:35358) 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 -- 172.21.15.116:0/403377056 learned_addr learned my addr 172.21.15.116:0/403377056 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291412640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c06ebf0 0x7fa28c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa290c11640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c0b2db0 0x7fa28c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 -- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c0b2db0 msgr2=0x7fa28c155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c0b2db0 0x7fa28c155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 -- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c06ebf0 msgr2=0x7fa28c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c06ebf0 0x7fa28c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 -- 172.21.15.116:0/403377056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288009400 con 0x7fa28c15dd90 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa291c13640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 0x7fa28c155a70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa284036040 tx=0x7fa284004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284009510 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28403e450 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28403e800 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.264+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa284009b30 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.265+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c1f3940 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.266+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c067180 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.266+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa28403e960 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.266+0000 7fa281ffb640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa27006bc70 0x7fa27006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.267+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6850+0+0 (secure 0 0 0) 0x7fa2840ad680 con 0x7fa28c15dd90 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.267+0000 7fa291412640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa27006bc70 0x7fa27006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:36.261 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.267+0000 7fa291412640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa27006bc70 0x7fa27006e130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa27c007fd0 tx=0x7fa27c007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:36.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.271+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa284077b20 con 0x7fa28c15dd90 2024-02-18T11:48:36.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:36.479+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa28c074520 con 0x7fa28c15dd90 2024-02-18T11:48:37.139 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.146+0000 7fa281ffb640 1 -- 172.21.15.116:0/403377056 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa28407b1e0 con 0x7fa28c15dd90 2024-02-18T11:48:37.141 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa27006bc70 msgr2=0x7fa27006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa27006bc70 0x7fa27006e130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa27c007fd0 tx=0x7fa27c007de0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 msgr2=0x7fa28c155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 0x7fa28c155a70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa284036040 tx=0x7fa284004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 shutdown_connections 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa27006bc70 0x7fa27006e130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c15dd90 0x7fa28c155a70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa28c0b2db0 0x7fa28c155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 --2- 172.21.15.116:0/403377056 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa28c06ebf0 0x7fa28c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 >> 172.21.15.116:0/403377056 conn(0x7fa28c0cfd80 msgr2=0x7fa28c0b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 shutdown_connections 2024-02-18T11:48:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.149+0000 7fa293676640 1 -- 172.21.15.116:0/403377056 wait complete. 2024-02-18T11:48:37.154 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-18T11:48:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.310+0000 7ff8fdb0c640 1 Processor -- start 2024-02-18T11:48:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.310+0000 7ff8fdb0c640 1 -- start start 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8fdb0c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8f80711c0 0x7ff8f8071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8fdb0c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8f815da30 0x7ff8f815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8fdb0c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f815c960 con 0x7ff8f8145230 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8fdb0c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f815d050 con 0x7ff8f815da30 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8fdb0c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f8145610 con 0x7ff8f80711c0 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8f815da30 0x7ff8f815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8f815da30 0x7ff8f815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35368/0 (socket says 172.21.15.116:35368) 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 -- 172.21.15.116:0/2096435127 learned_addr learned my addr 172.21.15.116:0/2096435127 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f6ffd640 1 --2- 172.21.15.116:0/2096435127 >> v2:172.21.15.143:3300/0 conn(0x7ff8f80711c0 0x7ff8f8071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 -- 172.21.15.116:0/2096435127 >> v2:172.21.15.143:3300/0 conn(0x7ff8f80711c0 msgr2=0x7ff8f8071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 --2- 172.21.15.116:0/2096435127 >> v2:172.21.15.143:3300/0 conn(0x7ff8f80711c0 0x7ff8f8071650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 -- 172.21.15.116:0/2096435127 >> v1:172.21.15.143:6789/0 conn(0x7ff8f8145230 legacy=0x7ff8f815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.311+0000 7ff8f7fff640 1 -- 172.21.15.116:0/2096435127 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f815cc40 con 0x7ff8f815da30 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.312+0000 7ff8f7fff640 1 --2- 172.21.15.116:0/2096435127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f815da30 0x7ff8f815de60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80cd540 tx=0x7ff8f002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3978e7117cce9e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.312+0000 7ff8f67fc640 1 -- 172.21.15.116:0/2096435127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f002cb20 con 0x7ff8f815da30 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.312+0000 7ff8f67fc640 1 -- 172.21.15.116:0/2096435127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f002cc80 con 0x7ff8f815da30 2024-02-18T11:48:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.312+0000 7ff8f67fc640 1 -- 172.21.15.116:0/2096435127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f00343f0 con 0x7ff8f815da30 2024-02-18T11:48:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2096435127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f815da30 msgr2=0x7ff8f815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2096435127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f815da30 0x7ff8f815de60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80cd540 tx=0x7ff8f002bf80 comp rx=0 tx=0).stop 2024-02-18T11:48:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2096435127 shutdown_connections 2024-02-18T11:48:37.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2096435127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f815da30 0x7ff8f815de60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2096435127 >> v2:172.21.15.143:3300/0 conn(0x7ff8f80711c0 0x7ff8f8071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2096435127 >> 172.21.15.116:0/2096435127 conn(0x7ff8f8067000 msgr2=0x7ff8f806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2096435127 shutdown_connections 2024-02-18T11:48:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.313+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2096435127 wait complete. 2024-02-18T11:48:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.314+0000 7ff8fdb0c640 1 Processor -- start 2024-02-18T11:48:37.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.314+0000 7ff8fdb0c640 1 -- start start 2024-02-18T11:48:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8fdb0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 0x7ff8f806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8fdb0c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f8145230 0x7ff8f813ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8fdb0c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff8f815da30 0x7ff8f813f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:37.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8fdb0c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80aa350 con 0x7ff8f80711c0 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8fdb0c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80a9740 con 0x7ff8f8145230 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8fdb0c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80a9a20 con 0x7ff8f815da30 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 0x7ff8f806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 0x7ff8f806d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:35380/0 (socket says 172.21.15.116:35380) 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 -- 172.21.15.116:0/2713862210 learned_addr learned my addr 172.21.15.116:0/2713862210 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f7fff640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff8f815da30 0x7ff8f813f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f6ffd640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f8145230 0x7ff8f813ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 -- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff8f815da30 msgr2=0x7ff8f813f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff8f815da30 0x7ff8f813f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 -- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f8145230 msgr2=0x7ff8f813ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f8145230 0x7ff8f813ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 -- 172.21.15.116:0/2713862210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0009b30 con 0x7ff8f80711c0 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f7fff640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff8f815da30 0x7ff8f813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:37.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f6ffd640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f8145230 0x7ff8f813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8f77fe640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 0x7ff8f806d470 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c400 tx=0x7ff8e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.315+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80052d0 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.316+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8005450 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.316+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8007260 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.316+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f80a9d00 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.316+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f806d9b0 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.317+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f80aa500 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.317+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ff8e80073c0 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.318+0000 7ff8dffff640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff8e406bc70 0x7ff8e406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.318+0000 7ff8f6ffd640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff8e406bc70 0x7ff8e406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.318+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6850+0+0 (secure 0 0 0) 0x7ff8e808a020 con 0x7ff8f80711c0 2024-02-18T11:48:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.318+0000 7ff8f6ffd640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff8e406bc70 0x7ff8e406e130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80ce170 tx=0x7ff8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:37.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.322+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8f80aa500 con 0x7ff8f80711c0 2024-02-18T11:48:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:37.494+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8f806baf0 con 0x7ff8f80711c0 2024-02-18T11:48:38.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.148+0000 7ff8dffff640 1 -- 172.21.15.116:0/2713862210 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff8e8057ec0 con 0x7ff8f80711c0 2024-02-18T11:48:38.140 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.151+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff8e406bc70 msgr2=0x7ff8e406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.151+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff8e406bc70 0x7ff8e406e130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80ce170 tx=0x7ff8ec008040 comp rx=0 tx=0).stop 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.151+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 msgr2=0x7ff8f806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.151+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 0x7ff8f806d470 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff8e800c400 tx=0x7ff8e800c8c0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 shutdown_connections 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ff8e406bc70 0x7ff8e406e130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff8f815da30 0x7ff8f813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8f8145230 0x7ff8f813ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 --2- 172.21.15.116:0/2713862210 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80711c0 0x7ff8f806d470 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 >> 172.21.15.116:0/2713862210 conn(0x7ff8f8067000 msgr2=0x7ff8f8143060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:38.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 shutdown_connections 2024-02-18T11:48:38.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.152+0000 7ff8fdb0c640 1 -- 172.21.15.116:0/2713862210 wait complete. 2024-02-18T11:48:38.157 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-18T11:48:38.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.271+0000 7fa89d3bc640 1 Processor -- start 2024-02-18T11:48:38.263 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.271+0000 7fa89d3bc640 1 -- start start 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa89d3bc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa89815dbf0 0x7fa8981617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa89d3bc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8980b9370 0x7fa8980ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa89d3bc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8980cec80 0x7fa89806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa89d3bc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980cf110 con 0x7fa8980cec80 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa89d3bc640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980d1ee0 con 0x7fa8980b9370 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa89d3bc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa898161dc0 con 0x7fa89815dbf0 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8980cec80 0x7fa89806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8980cec80 0x7fa89806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36680/0 (socket says 172.21.15.116:36680) 2024-02-18T11:48:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 -- 172.21.15.116:0/924868810 learned_addr learned my addr 172.21.15.116:0/924868810 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa896ffd640 1 --2- 172.21.15.116:0/924868810 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa89815dbf0 0x7fa8981617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8967fc640 1 --2- 172.21.15.116:0/924868810 >> v2:172.21.15.143:3300/0 conn(0x7fa8980b9370 0x7fa8980ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 -- 172.21.15.116:0/924868810 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa89815dbf0 msgr2=0x7fa8981617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 --2- 172.21.15.116:0/924868810 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa89815dbf0 0x7fa8981617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 -- 172.21.15.116:0/924868810 >> v2:172.21.15.143:3300/0 conn(0x7fa8980b9370 msgr2=0x7fa8980ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 --2- 172.21.15.116:0/924868810 >> v2:172.21.15.143:3300/0 conn(0x7fa8980b9370 0x7fa8980ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.272+0000 7fa8977fe640 1 -- 172.21.15.116:0/924868810 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8981620a0 con 0x7fa8980cec80 2024-02-18T11:48:38.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.273+0000 7fa8977fe640 1 --2- 172.21.15.116:0/924868810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa89806c210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa898068310 tx=0x7fa89002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6a0bb1d0a6f96b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:38.267 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.273+0000 7fa895ffb640 1 -- 172.21.15.116:0/924868810 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89002ca70 con 0x7fa8980cec80 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.273+0000 7fa895ffb640 1 -- 172.21.15.116:0/924868810 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89002cbd0 con 0x7fa8980cec80 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.273+0000 7fa895ffb640 1 -- 172.21.15.116:0/924868810 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900343f0 con 0x7fa8980cec80 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 -- 172.21.15.116:0/924868810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 msgr2=0x7fa89806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/924868810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa89806c210 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa898068310 tx=0x7fa89002bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 -- 172.21.15.116:0/924868810 shutdown_connections 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/924868810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa89806c210 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/924868810 >> v2:172.21.15.143:3300/0 conn(0x7fa8980b9370 0x7fa8980ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/924868810 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa89815dbf0 0x7fa8981617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 -- 172.21.15.116:0/924868810 >> 172.21.15.116:0/924868810 conn(0x7fa8980b5480 msgr2=0x7fa8980b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 -- 172.21.15.116:0/924868810 shutdown_connections 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.274+0000 7fa89d3bc640 1 -- 172.21.15.116:0/924868810 wait complete. 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.275+0000 7fa89d3bc640 1 Processor -- start 2024-02-18T11:48:38.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.275+0000 7fa89d3bc640 1 -- start start 2024-02-18T11:48:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa89d3bc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8980b9370 0x7fa898160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa89d3bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa8981547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa89d3bc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa89815dbf0 0x7fa898154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa89d3bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898152b70 con 0x7fa8980cec80 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa89d3bc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89815ec60 con 0x7fa89815dbf0 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa89d3bc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89815f190 con 0x7fa8980b9370 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa8981547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa8981547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36696/0 (socket says 172.21.15.116:36696) 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 -- 172.21.15.116:0/2985346637 learned_addr learned my addr 172.21.15.116:0/2985346637 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 -- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8980b9370 msgr2=0x7fa898160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8980b9370 0x7fa898160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8977fe640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa89815dbf0 0x7fa898154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 -- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa89815dbf0 msgr2=0x7fa898154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa89815dbf0 0x7fa898154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 -- 172.21.15.116:0/2985346637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa890009b30 con 0x7fa8980cec80 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.276+0000 7fa8967fc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa8981547b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00d830 tx=0x7fa88c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.277+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0096a0 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.277+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c009800 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.277+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89815e150 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.277+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c009b90 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.277+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89815eeb0 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.278+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa898067840 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.278+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa88c014410 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.278+0000 7fa87f7fe640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa88406bc70 0x7fa88406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.279+0000 7fa896ffd640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa88406bc70 0x7fa88406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.279+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6412+0+0 (secure 0 0 0) 0x7fa88c08b030 con 0x7fa8980cec80 2024-02-18T11:48:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.279+0000 7fa896ffd640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa88406bc70 0x7fa88406e130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa888006fd0 tx=0x7fa888008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:38.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.283+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa88c0555e0 con 0x7fa8980cec80 2024-02-18T11:48:38.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:38.456+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa898074520 con 0x7fa8980cec80 2024-02-18T11:48:39.147 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.155+0000 7fa87f7fe640 1 -- 172.21.15.116:0/2985346637 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa88c00b440 con 0x7fa8980cec80 2024-02-18T11:48:39.147 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa88406bc70 msgr2=0x7fa88406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa88406bc70 0x7fa88406e130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa888006fd0 tx=0x7fa888008040 comp rx=0 tx=0).stop 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 msgr2=0x7fa8981547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa8981547b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00d830 tx=0x7fa88c00dcf0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 shutdown_connections 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa88406bc70 0x7fa88406e130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa89815dbf0 0x7fa898154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8980cec80 0x7fa8981547b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 --2- 172.21.15.116:0/2985346637 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8980b9370 0x7fa898160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 >> 172.21.15.116:0/2985346637 conn(0x7fa8980b5480 msgr2=0x7fa8980b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:39.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.157+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 shutdown_connections 2024-02-18T11:48:39.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.158+0000 7fa89d3bc640 1 -- 172.21.15.116:0/2985346637 wait complete. 2024-02-18T11:48:39.168 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-18T11:48:39.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.281+0000 7f5d6330c640 1 Processor -- start 2024-02-18T11:48:39.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.281+0000 7f5d6330c640 1 -- start start 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d6330c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d5c0cee50 0x7f5d5c168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d6330c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c0d1ee0 con 0x7f5d5c15dbf0 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d6330c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5c161dc0 con 0x7f5d5c1617e0 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d6330c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5c1620a0 con 0x7f5d5c0cee50 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d618a9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d5c0cee50 0x7f5d5c168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d610a8640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5d5c15dbf0 0x7f5d5c0b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55774/0 (socket says 172.21.15.116:55774) 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.282+0000 7f5d610a8640 1 -- 172.21.15.116:0/165602463 learned_addr learned my addr 172.21.15.116:0/165602463 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:39.275 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.283+0000 7f5d618a9640 1 -- 172.21.15.116:0/165602463 >> v1:172.21.15.143:6789/0 conn(0x7f5d5c1617e0 legacy=0x7f5d5c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:39.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.283+0000 7f5d618a9640 1 -- 172.21.15.116:0/165602463 >> v1:172.21.15.116:6789/0 conn(0x7f5d5c15dbf0 legacy=0x7f5d5c0b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:39.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.283+0000 7f5d53fff640 1 -- 172.21.15.116:0/165602463 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525551490 0 0) 0x7f5d5c0d1ee0 con 0x7f5d5c15dbf0 2024-02-18T11:48:39.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.283+0000 7f5d618a9640 1 -- 172.21.15.116:0/165602463 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c162380 con 0x7f5d5c0cee50 2024-02-18T11:48:39.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.283+0000 7f5d618a9640 1 --2- 172.21.15.116:0/165602463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c0cee50 0x7f5d5c168b10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097a0 tx=0x7f5d5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d8fe3a44d4e9b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:39.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d53fff640 1 -- 172.21.15.116:0/165602463 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5402e070 con 0x7f5d5c0cee50 2024-02-18T11:48:39.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d53fff640 1 -- 172.21.15.116:0/165602463 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540315a0 con 0x7f5d5c0cee50 2024-02-18T11:48:39.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d53fff640 1 -- 172.21.15.116:0/165602463 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54031930 con 0x7f5d5c0cee50 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 -- 172.21.15.116:0/165602463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c0cee50 msgr2=0x7f5d5c168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 --2- 172.21.15.116:0/165602463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c0cee50 0x7f5d5c168b10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097a0 tx=0x7f5d5402b840 comp rx=0 tx=0).stop 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 -- 172.21.15.116:0/165602463 shutdown_connections 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 --2- 172.21.15.116:0/165602463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c0cee50 0x7f5d5c168b10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 -- 172.21.15.116:0/165602463 >> 172.21.15.116:0/165602463 conn(0x7f5d5c0b5480 msgr2=0x7f5d5c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 -- 172.21.15.116:0/165602463 shutdown_connections 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.284+0000 7f5d6330c640 1 -- 172.21.15.116:0/165602463 wait complete. 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.285+0000 7f5d6330c640 1 Processor -- start 2024-02-18T11:48:39.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.285+0000 7f5d6330c640 1 -- start start 2024-02-18T11:48:39.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d6330c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d5c0cee50 0x7f5d5c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:39.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d6330c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d5c15dbf0 0x7f5d5c1f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:39.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d6330c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 0x7f5d5c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:39.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d6330c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c14e3f0 con 0x7f5d5c1617e0 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d6330c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1f7690 con 0x7f5d5c15dbf0 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d6330c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1f7970 con 0x7f5d5c0cee50 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 0x7f5d5c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 0x7f5d5c156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36712/0 (socket says 172.21.15.116:36712) 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 -- 172.21.15.116:0/1360172039 learned_addr learned my addr 172.21.15.116:0/1360172039 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d610a8640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d5c0cee50 0x7f5d5c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d608a7640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d5c15dbf0 0x7f5d5c1f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:39.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 -- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d5c0cee50 msgr2=0x7f5d5c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d5c0cee50 0x7f5d5c14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 -- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d5c15dbf0 msgr2=0x7f5d5c1f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d5c15dbf0 0x7f5d5c1f6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d618a9640 1 -- 172.21.15.116:0/1360172039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d54009400 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.286+0000 7f5d610a8640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d5c0cee50 0x7f5d5c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.287+0000 7f5d618a9640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 0x7f5d5c156090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097a0 tx=0x7f5d540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.287+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54032840 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.287+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540329c0 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.287+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c15e150 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.287+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d54032d70 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.287+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c150450 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.288+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c067840 con 0x7f5d5c1617e0 2024-02-18T11:48:39.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.288+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f5d540453f0 con 0x7f5d5c1617e0 2024-02-18T11:48:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.289+0000 7f5d51ffb640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d4006bc70 0x7f5d4006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.289+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5d540ac810 con 0x7f5d5c1617e0 2024-02-18T11:48:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.289+0000 7f5d610a8640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d4006bc70 0x7f5d4006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:39.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.289+0000 7f5d610a8640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d4006bc70 0x7f5d4006e130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d580012e0 tx=0x7f5d58007f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:39.285 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.293+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d54076ff0 con 0x7f5d5c1617e0 2024-02-18T11:48:39.460 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:39.467+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d5c074520 con 0x7f5d5c1617e0 2024-02-18T11:48:40.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.159+0000 7f5d51ffb640 1 -- 172.21.15.116:0/1360172039 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f5d5407a6b0 con 0x7f5d5c1617e0 2024-02-18T11:48:40.152 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data_ec' removed 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.163+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d4006bc70 msgr2=0x7f5d4006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.163+0000 7f5d6330c640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d4006bc70 0x7f5d4006e130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d580012e0 tx=0x7f5d58007f90 comp rx=0 tx=0).stop 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.163+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 msgr2=0x7f5d5c156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.163+0000 7f5d6330c640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 0x7f5d5c156090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5d540097a0 tx=0x7f5d540046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.163+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 shutdown_connections 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5d4006bc70 0x7f5d4006e130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d5c1617e0 0x7f5d5c156090 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.156 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d5c15dbf0 0x7f5d5c1f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 --2- 172.21.15.116:0/1360172039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5d5c0cee50 0x7f5d5c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 >> 172.21.15.116:0/1360172039 conn(0x7f5d5c0b5480 msgr2=0x7f5d5c1584c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:40.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 shutdown_connections 2024-02-18T11:48:40.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.164+0000 7f5d6330c640 1 -- 172.21.15.116:0/1360172039 wait complete. 2024-02-18T11:48:40.171 INFO:tasks.ceph.mds.a:Restarting daemon 2024-02-18T11:48:40.171 INFO:tasks.ceph.mds.a:Stopping old one... 2024-02-18T11:48:40.171 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-02-18T11:48:40.171 INFO:teuthology.orchestra.run:waiting for 300 2024-02-18T11:48:40.171 INFO:tasks.ceph.mds.c:Restarting daemon 2024-02-18T11:48:40.171 INFO:tasks.ceph.mds.c:Stopping old one... 2024-02-18T11:48:40.172 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-02-18T11:48:40.172 INFO:teuthology.orchestra.run:waiting for 300 2024-02-18T11:48:40.172 INFO:tasks.ceph.mds.b:Restarting daemon 2024-02-18T11:48:40.172 INFO:tasks.ceph.mds.b:Stopping old one... 2024-02-18T11:48:40.172 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-02-18T11:48:40.172 INFO:teuthology.orchestra.run:waiting for 300 2024-02-18T11:48:40.172 INFO:tasks.ceph.mds.d:Restarting daemon 2024-02-18T11:48:40.173 INFO:tasks.ceph.mds.d:Stopping old one... 2024-02-18T11:48:40.173 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-02-18T11:48:40.173 INFO:teuthology.orchestra.run:waiting for 300 2024-02-18T11:48:40.183 INFO:tasks.ceph.mds.c:Stopped 2024-02-18T11:48:40.183 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-02-18T11:48:40.183 INFO:tasks.ceph.mds.a:Stopped 2024-02-18T11:48:40.184 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-02-18T11:48:40.185 INFO:tasks.ceph.mds.b:Stopped 2024-02-18T11:48:40.185 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-02-18T11:48:40.186 INFO:tasks.ceph.mds.d:Stopped 2024-02-18T11:48:40.186 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-02-18T11:48:40.225 INFO:tasks.ceph.mds.c:Started 2024-02-18T11:48:40.226 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:40.227 INFO:tasks.ceph.mds.a:Started 2024-02-18T11:48:40.227 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:40.228 INFO:tasks.ceph.mds.b:Started 2024-02-18T11:48:40.228 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:40.229 INFO:tasks.ceph.mds.d:Started 2024-02-18T11:48:40.229 DEBUG:teuthology.parallel:result is None 2024-02-18T11:48:40.229 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-02-18T11:48:40.305 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2024-02-18T11:48:40.306 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2024-02-18T11:48:40.311 INFO:tasks.ceph.mds.b.smithi143.stdout:starting mds.b at 2024-02-18T11:48:40.311 INFO:tasks.ceph.mds.d.smithi143.stdout:starting mds.d at 2024-02-18T11:48:40.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.347+0000 7fa2d97e2640 1 Processor -- start 2024-02-18T11:48:40.339 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.347+0000 7fa2d97e2640 1 -- start start 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d97e2640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa2d415dba0 0x7fa2d4161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d97e2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d40d21b0 con 0x7fa2d40b93b0 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d97e2640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d40d1e60 con 0x7fa2d40cef00 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d97e2640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d4162050 con 0x7fa2d415dba0 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2cbfff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa2d40b93b0 0x7fa2d40ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55794/0 (socket says 172.21.15.116:55794) 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2cbfff640 1 -- 172.21.15.116:0/2679546368 learned_addr learned my addr 172.21.15.116:0/2679546368 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146767467 0 0) 0x7fa2d40d21b0 con 0x7fa2d40b93b0 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/2679546368 >> v2:172.21.15.143:3300/0 conn(0x7fa2d415dba0 0x7fa2d4161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c0003520 con 0x7fa2d40b93b0 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 658479688 0 0) 0x7fa2c0003520 con 0x7fa2d40b93b0 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 >> v2:172.21.15.143:3300/0 conn(0x7fa2d415dba0 msgr2=0x7fa2d4161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 --2- 172.21.15.116:0/2679546368 >> v2:172.21.15.143:3300/0 conn(0x7fa2d415dba0 0x7fa2d4161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 >> v1:172.21.15.143:6789/0 conn(0x7fa2d40cef00 legacy=0x7fa2d40d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d4162330 con 0x7fa2d40b93b0 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fa2bc001670 con 0x7fa2d40b93b0 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 >> v1:172.21.15.116:6789/0 conn(0x7fa2d40b93b0 legacy=0x7fa2d40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c0003970 0x7fa2c0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0004390 0x7fa2c0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0005060 0x7fa2c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4162050 con 0x7fa2c0005060 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000bfa0 con 0x7fa2c0003970 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c000c160 con 0x7fa2c0004390 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.348+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0005060 0x7fa2c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2cbfff640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0004390 0x7fa2c0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c0003970 0x7fa2c0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d37fe640 1 -- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0004390 msgr2=0x7fa2c0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0004390 0x7fa2c0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d37fe640 1 -- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c0003970 msgr2=0x7fa2c0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c0003970 0x7fa2c0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d37fe640 1 -- 172.21.15.116:0/2679546368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0003520 con 0x7fa2c0005060 2024-02-18T11:48:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2c0003970 0x7fa2c0003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:40.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.349+0000 7fa2d37fe640 1 -- 172.21.15.116:0/2679546368 reap_dead start 2024-02-18T11:48:40.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0005060 0x7fa2c000b980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001e80 tx=0x7fa2c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba484a15d6d178a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:40.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003b70 con 0x7fa2c0005060 2024-02-18T11:48:40.344 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc003cd0 con 0x7fa2c0005060 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d27fc640 1 -- 172.21.15.116:0/2679546368 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc005310 con 0x7fa2c0005060 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d97e2640 1 -- 172.21.15.116:0/2679546368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0005060 msgr2=0x7fa2c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0005060 0x7fa2c000b980 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001e80 tx=0x7fa2c00057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d97e2640 1 -- 172.21.15.116:0/2679546368 shutdown_connections 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/2679546368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2c0005060 0x7fa2c000b980 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.350+0000 7fa2d97e2640 1 -- 172.21.15.116:0/2679546368 >> 172.21.15.116:0/2679546368 conn(0x7fa2d40d04a0 msgr2=0x7fa2d40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.351+0000 7fa2d97e2640 1 -- 172.21.15.116:0/2679546368 shutdown_connections 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.351+0000 7fa2d97e2640 1 -- 172.21.15.116:0/2679546368 wait complete. 2024-02-18T11:48:40.345 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.353+0000 7fa2d97e2640 1 Processor -- start 2024-02-18T11:48:40.346 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.354+0000 7fa2d97e2640 1 -- start start 2024-02-18T11:48:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.355+0000 7fa2d97e2640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0005060 0x7fa2d41f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.355+0000 7fa2d97e2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d41f3a10 0x7fa2d41f0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.355+0000 7fa2d97e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 0x7fa2d40c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.355+0000 7fa2d97e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40c8f80 con 0x7fa2d40c85b0 2024-02-18T11:48:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.355+0000 7fa2d97e2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41f7db0 con 0x7fa2d41f3a10 2024-02-18T11:48:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.355+0000 7fa2d97e2640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41f77f0 con 0x7fa2c0005060 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 0x7fa2d40c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 0x7fa2d40c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36760/0 (socket says 172.21.15.116:36760) 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 -- 172.21.15.116:0/419917112 learned_addr learned my addr 172.21.15.116:0/419917112 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0005060 0x7fa2d41f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 -- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0005060 msgr2=0x7fa2d41f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0005060 0x7fa2d41f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 -- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d41f3a10 msgr2=0x7fa2d41f0c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2cbfff640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d41f3a10 0x7fa2d41f0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d41f3a10 0x7fa2d41f0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.356+0000 7fa2d37fe640 1 -- 172.21.15.116:0/419917112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d41f7ad0 con 0x7fa2d40c85b0 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.357+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0005060 0x7fa2d41f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.357+0000 7fa2d37fe640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 0x7fa2d40c8a40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001fa0 tx=0x7fa2cc001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.357+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc007df0 con 0x7fa2d40c85b0 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.358+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc003e40 con 0x7fa2d40c85b0 2024-02-18T11:48:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.358+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0386d0 con 0x7fa2d40c85b0 2024-02-18T11:48:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.359+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d41f4b60 con 0x7fa2d40c85b0 2024-02-18T11:48:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.359+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d41f7430 con 0x7fa2d40c85b0 2024-02-18T11:48:40.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.360+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d41f49c0 con 0x7fa2d40c85b0 2024-02-18T11:48:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.360+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa2cc03f030 con 0x7fa2d40c85b0 2024-02-18T11:48:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.361+0000 7fa2cb7fe640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa2b406bc70 0x7fa2b406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.361+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa2b406bc70 0x7fa2b406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:40.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.361+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa2cc0abd60 con 0x7fa2d40c85b0 2024-02-18T11:48:40.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.362+0000 7fa2d2ffd640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa2b406bc70 0x7fa2b406e130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009b40 tx=0x7fa2c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:40.358 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.367+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2d41f49c0 con 0x7fa2d40c85b0 2024-02-18T11:48:40.543 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:40.551+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa2d41f5080 con 0x7fa2d40c85b0 2024-02-18T11:48:41.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.169+0000 7fa2cb7fe640 1 -- 172.21.15.116:0/419917112 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa2d41f5080 con 0x7fa2d40c85b0 2024-02-18T11:48:41.162 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa2b406bc70 msgr2=0x7fa2b406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa2b406bc70 0x7fa2b406e130 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009b40 tx=0x7fa2c4008040 comp rx=0 tx=0).stop 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 msgr2=0x7fa2d40c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 0x7fa2d40c8a40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001fa0 tx=0x7fa2cc001440 comp rx=0 tx=0).stop 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 shutdown_connections 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa2b406bc70 0x7fa2b406e130 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2d40c85b0 0x7fa2d40c8a40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa2d41f3a10 0x7fa2d41f0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 --2- 172.21.15.116:0/419917112 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa2c0005060 0x7fa2d41f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 >> 172.21.15.116:0/419917112 conn(0x7fa2d40d04a0 msgr2=0x7fa2d40b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 shutdown_connections 2024-02-18T11:48:41.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.172+0000 7fa2d97e2640 1 -- 172.21.15.116:0/419917112 wait complete. 2024-02-18T11:48:41.178 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-02-18T11:48:41.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.288+0000 7f3487064640 1 Processor -- start 2024-02-18T11:48:41.280 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.288+0000 7f3487064640 1 -- start start 2024-02-18T11:48:41.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.288+0000 7f3487064640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f34800b1960 0x7f348006d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.281 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3487064640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34800cdff0 con 0x7f348015ae40 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3487064640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348015b3e0 con 0x7f34800b0210 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3487064640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34800b1070 con 0x7f34800b1960 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f347ffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f348015ae40 0x7f34800cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55798/0 (socket says 172.21.15.116:55798) 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f347ffff640 1 -- 172.21.15.116:0/746134769 learned_addr learned my addr 172.21.15.116:0/746134769 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3485601640 1 --2- 172.21.15.116:0/746134769 >> v2:172.21.15.116:3300/0 conn(0x7f34800b1960 0x7f348006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f347f7fe640 1 -- 172.21.15.116:0/746134769 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799180086 0 0) 0x7f34800cdff0 con 0x7f348015ae40 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f347f7fe640 1 -- 172.21.15.116:0/746134769 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003520 con 0x7f348015ae40 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3485601640 1 -- 172.21.15.116:0/746134769 >> v1:172.21.15.143:6789/0 conn(0x7f34800b0210 legacy=0x7f34800b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3485601640 1 -- 172.21.15.116:0/746134769 >> v1:172.21.15.116:6789/0 conn(0x7f348015ae40 legacy=0x7f34800cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.289+0000 7f3485601640 1 -- 172.21.15.116:0/746134769 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800ceb50 con 0x7f34800b1960 2024-02-18T11:48:41.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.290+0000 7f3485601640 1 --2- 172.21.15.116:0/746134769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34800b1960 0x7f348006d200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f34680097a0 tx=0x7f346802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f10cbdc932f7397 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:41.283 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.290+0000 7f347f7fe640 1 -- 172.21.15.116:0/746134769 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468030610 con 0x7f34800b1960 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.290+0000 7f347f7fe640 1 -- 172.21.15.116:0/746134769 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468030770 con 0x7f34800b1960 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.290+0000 7f347f7fe640 1 -- 172.21.15.116:0/746134769 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468030b00 con 0x7f34800b1960 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 -- 172.21.15.116:0/746134769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34800b1960 msgr2=0x7f348006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 --2- 172.21.15.116:0/746134769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34800b1960 0x7f348006d200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f34680097a0 tx=0x7f346802b960 comp rx=0 tx=0).stop 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 -- 172.21.15.116:0/746134769 shutdown_connections 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 --2- 172.21.15.116:0/746134769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34800b1960 0x7f348006d200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 -- 172.21.15.116:0/746134769 >> 172.21.15.116:0/746134769 conn(0x7f3480066f20 msgr2=0x7f348006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 -- 172.21.15.116:0/746134769 shutdown_connections 2024-02-18T11:48:41.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.291+0000 7f3487064640 1 -- 172.21.15.116:0/746134769 wait complete. 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.292+0000 7f3487064640 1 Processor -- start 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.292+0000 7f3487064640 1 -- start start 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34800b0210 0x7f34801438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34800b1960 0x7f3480144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 0x7f3480144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480154ab0 con 0x7f348015ae40 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348014b800 con 0x7f34800b0210 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3480156e90 con 0x7f34800b1960 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 0x7f3480144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 0x7f3480144d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36766/0 (socket says 172.21.15.116:36766) 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 -- 172.21.15.116:0/42844554 learned_addr learned my addr 172.21.15.116:0/42844554 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 -- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34800b1960 msgr2=0x7f3480144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34800b1960 0x7f3480144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 -- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34800b0210 msgr2=0x7f34801438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3484e00640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34800b0210 0x7f34801438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34800b0210 0x7f34801438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 -- 172.21.15.116:0/42844554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468009400 con 0x7f348015ae40 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3485601640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 0x7f3480144d60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f34680098c0 tx=0x7f3468004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346803e040 con 0x7f348015ae40 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468039c10 con 0x7f348015ae40 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3480157170 con 0x7f348015ae40 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468030e10 con 0x7f348015ae40 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.293+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3480157730 con 0x7f348015ae40 2024-02-18T11:48:41.291 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.295+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3480154c20 con 0x7f348015ae40 2024-02-18T11:48:41.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.295+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f346804d3b0 con 0x7f348015ae40 2024-02-18T11:48:41.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.295+0000 7f347d7fa640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f346006bc70 0x7f346006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.295+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f34680acb80 con 0x7f348015ae40 2024-02-18T11:48:41.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.295+0000 7f3484e00640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f346006bc70 0x7f346006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.292 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.296+0000 7f3484e00640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f346006bc70 0x7f346006e130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3470007fd0 tx=0x7f3470009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:41.293 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.299+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3480154c20 con 0x7f348015ae40 2024-02-18T11:48:41.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.534+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3480154bf0 con 0x7f348015ae40 2024-02-18T11:48:41.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.535+0000 7f347d7fa640 1 -- 172.21.15.116:0/42844554 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3468034090 con 0x7f348015ae40 2024-02-18T11:48:41.527 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:41.527 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlyhFMGxAAMeF+iwTSDh0X1A3ZXxuDTg==", 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.528 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlN/msJxAAJqlgK7WxiuEWdZrEG8v2RQ==", 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2024-02-18T11:48:41.529 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlNJexIhAAp4vwOjUHQEcuqo2D72RV+Q==", 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlBHwKLxAADXZv7IdT3E6Hhuv1dtwRZg==", 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.530 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAg7tFlXmZiEBAAfS+Eercp/1xgAcPRd14DoQ==", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.531 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAj7tFl+savDhAAicZUH6PgxZwa1O26SjOKhg==", 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.532 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAm7tFlYhrKDBAALKMstc0EPp/QQlFRwhxBwA==", 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAp7tFlmZ1TChAAVLXNxeffx0iciG6tBX+7JA==", 2024-02-18T11:48:41.533 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAv7tFlCltEERAA1CFMxClfgFn3GuPqDjkIWA==", 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.534 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAy7tFlzGTgEBAADx0h93HsLgqsh9nT3XThVQ==", 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.535 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA17tFllAVeDxAAOduu0ARW9LFB4aCb5f59Pg==", 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA47tFly22eDRAAlSlKL15kQi7t8ftXRZY6Yw==", 2024-02-18T11:48:41.536 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow profile osd", 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile osd", 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlVhFMMxAAcC+CBcaJjyYSC7rVBwyvOw==", 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-02-18T11:48:41.537 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAd7tFloBHcABAAXBVf3C+t7Ia28G5cwLYxGg==", 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-02-18T11:48:41.538 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlyCgUORAA3NQgD2krY5qBoJd79bNekg==", 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.539 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAd7tFleeykBhAAS4gRO+FydukZBSNIOL8gOA==", 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.540 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg==", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-mds", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA+7tFlMWlGBhAAYXXGC4AHjGVJdTX4TdWG5Q==", 2024-02-18T11:48:41.541 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-mds" 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-mgr", 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA+7tFlZXFGBhAA39hawHOZO76WcHDsMeyfwg==", 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-mgr" 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.542 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-osd", 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA+7tFlInhGBhAAhdaI7x2B9TU9VQ7TevncOQ==", 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-osd" 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rbd", 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA+7tFlZH5GBhAA+9OVzQe/Cb/NUxb9YlEfNg==", 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rbd" 2024-02-18T11:48:41.543 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA+7tFlBoVGBhAAzXltF4eWDYzYztiEfvIiVA==", 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rgw", 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQA+7tFlv4tGBhAA+zy2ndYKgERxTFK33Z0JGA==", 2024-02-18T11:48:41.544 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rgw" 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlXYfnFxAAoKC7TGSsy6LJIlhW35+8rg==", 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.545 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAc7tFlo3OFExAA02E+7DFvUakguPorwbhBUA==", 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-02-18T11:48:41.546 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f346006bc70 msgr2=0x7f346006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f346006bc70 0x7f346006e130 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3470007fd0 tx=0x7f3470009040 comp rx=0 tx=0).stop 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 msgr2=0x7f3480144d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 0x7f3480144d60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f34680098c0 tx=0x7f3468004730 comp rx=0 tx=0).stop 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 shutdown_connections 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f346006bc70 0x7f346006e130 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f348015ae40 0x7f3480144d60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f34800b1960 0x7f3480144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.550 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 --2- 172.21.15.116:0/42844554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34800b0210 0x7f34801438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 >> 172.21.15.116:0/42844554 conn(0x7f3480066f20 msgr2=0x7f34800b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:41.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 shutdown_connections 2024-02-18T11:48:41.551 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.537+0000 7f3487064640 1 -- 172.21.15.116:0/42844554 wait complete. 2024-02-18T11:48:41.551 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-02-18T11:48:41.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.660+0000 7fd8420df640 1 Processor -- start 2024-02-18T11:48:41.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.660+0000 7fd8420df640 1 -- start start 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd8420df640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd83c0711c0 0x7fd83c071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd8420df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 0x7fd83c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd8420df640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c15c960 con 0x7fd83c145230 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd8420df640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c15d050 con 0x7fd83c0711c0 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd8420df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c145610 con 0x7fd83c15da30 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd83b7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd83c145230 0x7fd83c15be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55806/0 (socket says 172.21.15.116:55806) 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd83b7fe640 1 -- 172.21.15.116:0/3189432786 learned_addr learned my addr 172.21.15.116:0/3189432786 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd83affd640 1 --2- 172.21.15.116:0/3189432786 >> v2:172.21.15.143:3300/0 conn(0x7fd83c0711c0 0x7fd83c071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd83bfff640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 0x7fd83c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3045331483 0 0) 0x7fd83c15c960 con 0x7fd83c145230 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.661+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd824003520 con 0x7fd83c145230 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 882152580 0 0) 0x7fd824003520 con 0x7fd83c145230 2024-02-18T11:48:41.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 msgr2=0x7fd83c15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 0x7fd83c15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 >> v2:172.21.15.143:3300/0 conn(0x7fd83c0711c0 msgr2=0x7fd83c071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 --2- 172.21.15.116:0/3189432786 >> v2:172.21.15.143:3300/0 conn(0x7fd83c0711c0 0x7fd83c071650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c15cc40 con 0x7fd83c145230 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fd82c001690 con 0x7fd83c145230 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 >> v1:172.21.15.116:6789/0 conn(0x7fd83c145230 legacy=0x7fd83c15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 0x7fd824003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd824003ef0 0x7fd8240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd824004d00 0x7fd824005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c15d050 con 0x7fd824004d00 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400b980 con 0x7fd824003ef0 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82400bac0 con 0x7fd83c15da30 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83affd640 1 --2- 172.21.15.116:0/3189432786 >> v2:172.21.15.143:3300/0 conn(0x7fd83c0711c0 0x7fd83c071650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:41.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83bfff640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd824004d00 0x7fd824005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83b7fe640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 0x7fd824003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83affd640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd824003ef0 0x7fd8240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83bfff640 1 -- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 msgr2=0x7fd824003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83bfff640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c15da30 0x7fd824003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83bfff640 1 -- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd824003ef0 msgr2=0x7fd8240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83bfff640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd824003ef0 0x7fd8240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.662+0000 7fd83bfff640 1 -- 172.21.15.116:0/3189432786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824003520 con 0x7fd824004d00 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd83bfff640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd824004d00 0x7fd824005180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd834009fc0 tx=0x7fd8240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590b01e8df2a2a49 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83402cc00 con 0x7fd824004d00 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83400f040 con 0x7fd824004d00 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd83a7fc640 1 -- 172.21.15.116:0/3189432786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834034440 con 0x7fd824004d00 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd8420df640 1 -- 172.21.15.116:0/3189432786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd824004d00 msgr2=0x7fd824005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd8420df640 1 --2- 172.21.15.116:0/3189432786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd824004d00 0x7fd824005180 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd834009fc0 tx=0x7fd8240057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.659 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.663+0000 7fd83bfff640 1 -- 172.21.15.116:0/3189432786 reap_dead start 2024-02-18T11:48:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.664+0000 7fd8420df640 1 -- 172.21.15.116:0/3189432786 shutdown_connections 2024-02-18T11:48:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.664+0000 7fd8420df640 1 -- 172.21.15.116:0/3189432786 >> 172.21.15.116:0/3189432786 conn(0x7fd83c067000 msgr2=0x7fd83c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.664+0000 7fd8420df640 1 -- 172.21.15.116:0/3189432786 shutdown_connections 2024-02-18T11:48:41.660 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.664+0000 7fd8420df640 1 -- 172.21.15.116:0/3189432786 wait complete. 2024-02-18T11:48:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.665+0000 7fd8420df640 1 Processor -- start 2024-02-18T11:48:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.665+0000 7fd8420df640 1 -- start start 2024-02-18T11:48:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 0x7fd83c0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c0b3b50 0x7fd83c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0ae030 0x7fd83c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0c6a20 con 0x7fd83c15b530 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0ae4b0 con 0x7fd83c0ae030 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b36c0 con 0x7fd83c0b3b50 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 0x7fd83c0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 0x7fd83c0b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36778/0 (socket says 172.21.15.116:36778) 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 -- 172.21.15.116:0/2901895796 learned_addr learned my addr 172.21.15.116:0/2901895796 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83bfff640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0ae030 0x7fd83c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 -- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c0b3b50 msgr2=0x7fd83c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c0b3b50 0x7fd83c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 -- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0ae030 msgr2=0x7fd83c0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0ae030 0x7fd83c0c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 -- 172.21.15.116:0/2901895796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0b2ab0 con 0x7fd83c15b530 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83bfff640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0ae030 0x7fd83c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd83b7fe640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 0x7fd83c0b4100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd82c007340 tx=0x7fd82c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:41.664 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c00f9d0 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c00fb50 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c0b2d90 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0133f0 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.666+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c0b3350 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.667+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c0b3510 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.668+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd82c013550 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.668+0000 7fd823fff640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82806bc70 0x7fd82806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.668+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd82c08a920 con 0x7fd83c15b530 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.668+0000 7fd83affd640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82806bc70 0x7fd82806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:41.665 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.669+0000 7fd83affd640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82806bc70 0x7fd82806e130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0aef70 tx=0x7fd830008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:41.666 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.672+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd82c0552c0 con 0x7fd83c15b530 2024-02-18T11:48:41.899 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.907+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd83c06baf0 con 0x7fd83c15b530 2024-02-18T11:48:41.900 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.909+0000 7fd823fff640 1 -- 172.21.15.116:0/2901895796 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd82c058980 con 0x7fd83c15b530 2024-02-18T11:48:41.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82806bc70 msgr2=0x7fd82806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82806bc70 0x7fd82806e130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0aef70 tx=0x7fd830008040 comp rx=0 tx=0).stop 2024-02-18T11:48:41.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 msgr2=0x7fd83c0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:41.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 0x7fd83c0b4100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd82c007340 tx=0x7fd82c007800 comp rx=0 tx=0).stop 2024-02-18T11:48:41.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 shutdown_connections 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd82806bc70 0x7fd82806e130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0ae030 0x7fd83c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd83c0b3b50 0x7fd83c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 --2- 172.21.15.116:0/2901895796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd83c15b530 0x7fd83c0b4100 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 >> 172.21.15.116:0/2901895796 conn(0x7fd83c067000 msgr2=0x7fd83c0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 shutdown_connections 2024-02-18T11:48:41.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:41.911+0000 7fd8420df640 1 -- 172.21.15.116:0/2901895796 wait complete. 2024-02-18T11:48:41.914 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-02-18T11:48:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.023+0000 7f497609d640 1 Processor -- start 2024-02-18T11:48:42.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.023+0000 7f497609d640 1 -- start start 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f497609d640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f497015dc90 0x7f49700ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f497609d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49700cf980 con 0x7f49700cf3e0 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f497609d640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497006ab60 con 0x7f497006f300 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f497609d640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497015e070 con 0x7f497015dc90 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496effd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f49700cf3e0 0x7f497006ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55818/0 (socket says 172.21.15.116:55818) 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496effd640 1 -- 172.21.15.116:0/989180673 learned_addr learned my addr 172.21.15.116:0/989180673 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496f7fe640 1 --2- 172.21.15.116:0/989180673 >> v2:172.21.15.143:3300/0 conn(0x7f497015dc90 0x7f49700ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475066988 0 0) 0x7f49700cf980 con 0x7f49700cf3e0 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4958003520 con 0x7f49700cf3e0 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383342209 0 0) 0x7f4958003520 con 0x7f49700cf3e0 2024-02-18T11:48:42.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 >> v2:172.21.15.143:3300/0 conn(0x7f497015dc90 msgr2=0x7f49700ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 --2- 172.21.15.116:0/989180673 >> v2:172.21.15.143:3300/0 conn(0x7f497015dc90 0x7f49700ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 >> v1:172.21.15.143:6789/0 conn(0x7f497006f300 legacy=0x7f49700b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497006ae40 con 0x7f49700cf3e0 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f4964001670 con 0x7f49700cf3e0 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 >> v1:172.21.15.116:6789/0 conn(0x7f49700cf3e0 legacy=0x7f497006ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958003970 0x7f4958003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49580047c0 0x7f4958004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4958005180 0x7f495800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497015e070 con 0x7f4958005180 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495800c050 con 0x7f49580047c0 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495800c210 con 0x7f4958003970 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.024+0000 7f496ffff640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4958005180 0x7f495800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496f7fe640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958003970 0x7f4958003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496effd640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49580047c0 0x7f4958004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 -- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958003970 msgr2=0x7f4958003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958003970 0x7f4958003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 -- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49580047c0 msgr2=0x7f4958004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49580047c0 0x7f4958004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 -- 172.21.15.116:0/989180673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4958003520 con 0x7f4958005180 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496f7fe640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958003970 0x7f4958003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496effd640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49580047c0 0x7f4958004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 -- 172.21.15.116:0/989180673 reap_dead start 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.025+0000 7f496ffff640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4958005180 0x7f495800b980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4968001e50 tx=0x7f49580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6210e7feffd6f31 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968003b70 con 0x7f4958005180 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968003d10 con 0x7f4958005180 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f496e7fc640 1 -- 172.21.15.116:0/989180673 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968005310 con 0x7f4958005180 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 -- 172.21.15.116:0/989180673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4958005180 msgr2=0x7f495800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4958005180 0x7f495800b980 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4968001e50 tx=0x7f49580057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 -- 172.21.15.116:0/989180673 shutdown_connections 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 --2- 172.21.15.116:0/989180673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4958005180 0x7f495800b980 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 -- 172.21.15.116:0/989180673 >> 172.21.15.116:0/989180673 conn(0x7f49700cfd80 msgr2=0x7f49700d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 -- 172.21.15.116:0/989180673 shutdown_connections 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.026+0000 7f497609d640 1 -- 172.21.15.116:0/989180673 wait complete. 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.027+0000 7f497609d640 1 Processor -- start 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.027+0000 7f497609d640 1 -- start start 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f497609d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958005180 0x7f497015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f497609d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 0x7f49700c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f497609d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49700c5260 0x7f49700aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f497609d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700c56e0 con 0x7f49701481a0 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f497609d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700ccbe0 con 0x7f49700c5260 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f497609d640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4970148a00 con 0x7f4958005180 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 0x7f49700c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.025 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 0x7f49700c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36802/0 (socket says 172.21.15.116:36802) 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 -- 172.21.15.116:0/3430890481 learned_addr learned my addr 172.21.15.116:0/3430890481 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496ffff640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49700c5260 0x7f49700aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496f7fe640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958005180 0x7f497015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 -- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958005180 msgr2=0x7f497015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958005180 0x7f497015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 -- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49700c5260 msgr2=0x7f49700aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49700c5260 0x7f49700aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.028+0000 7f496effd640 1 -- 172.21.15.116:0/3430890481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970148ca0 con 0x7f49701481a0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.029+0000 7f496effd640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 0x7f49700c4d20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f496400e7b0 tx=0x7f496400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.029+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400eec0 con 0x7f49701481a0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.029+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496400c870 con 0x7f49701481a0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.029+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4970148f80 con 0x7f49701481a0 2024-02-18T11:48:42.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.029+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400cc00 con 0x7f49701481a0 2024-02-18T11:48:42.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.029+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970149540 con 0x7f49701481a0 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.030+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4970067180 con 0x7f49701481a0 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.030+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f496401d460 con 0x7f49701481a0 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.031+0000 7f4957fff640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f495c06bc70 0x7f495c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.031+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f496408a650 con 0x7f49701481a0 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.031+0000 7f496f7fe640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f495c06bc70 0x7f495c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.031+0000 7f496f7fe640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f495c06bc70 0x7f495c06e130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4960009990 tx=0x7f4960008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.035+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4964054f60 con 0x7f49701481a0 2024-02-18T11:48:42.264 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.272+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4970149760 con 0x7f49701481a0 2024-02-18T11:48:42.266 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.274+0000 7f4957fff640 1 -- 172.21.15.116:0/3430890481 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4964058620 con 0x7f49701481a0 2024-02-18T11:48:42.268 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f495c06bc70 msgr2=0x7f495c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f495c06bc70 0x7f495c06e130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4960009990 tx=0x7f4960008040 comp rx=0 tx=0).stop 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 msgr2=0x7f49700c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 0x7f49700c4d20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f496400e7b0 tx=0x7f496400ec70 comp rx=0 tx=0).stop 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 shutdown_connections 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f495c06bc70 0x7f495c06e130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49700c5260 0x7f49700aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49701481a0 0x7f49700c4d20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 --2- 172.21.15.116:0/3430890481 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4958005180 0x7f497015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 >> 172.21.15.116:0/3430890481 conn(0x7f49700cfd80 msgr2=0x7f49700ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 shutdown_connections 2024-02-18T11:48:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.276+0000 7f497609d640 1 -- 172.21.15.116:0/3430890481 wait complete. 2024-02-18T11:48:42.279 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-02-18T11:48:42.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.389+0000 7ffa8598c640 1 Processor -- start 2024-02-18T11:48:42.382 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.389+0000 7ffa8598c640 1 -- start start 2024-02-18T11:48:42.383 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa8598c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa800ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa8598c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffa800cec80 0x7ffa800d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa8598c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa800d1ee0 con 0x7ffa8015dbf0 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa8598c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa800cf110 con 0x7ffa800cec80 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa8598c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80161dc0 con 0x7ffa801617e0 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa800ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7f7fe640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffa800cec80 0x7ffa800d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa800ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.116:45002/0 (socket says 172.21.15.116:45002) 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 -- 172.21.15.116:0/2727237971 learned_addr learned my addr 172.21.15.116:0/2727237971 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7effd640 1 --1- 172.21.15.116:0/2727237971 >> v1:172.21.15.143:6789/0 conn(0x7ffa8015dbf0 0x7ffa800b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:42856/0 (socket says 172.21.15.116:42856) 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 -- 172.21.15.116:0/2727237971 >> v2:172.21.15.143:3300/0 conn(0x7ffa800cec80 msgr2=0x7ffa800d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 --2- 172.21.15.116:0/2727237971 >> v2:172.21.15.143:3300/0 conn(0x7ffa800cec80 0x7ffa800d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 -- 172.21.15.116:0/2727237971 >> v1:172.21.15.143:6789/0 conn(0x7ffa8015dbf0 legacy=0x7ffa800b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7dffb640 1 -- 172.21.15.116:0/2727237971 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200427837 0 0) 0x7ffa800d1ee0 con 0x7ffa8015dbf0 2024-02-18T11:48:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7e7fc640 1 -- 172.21.15.116:0/2727237971 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa801620a0 con 0x7ffa801617e0 2024-02-18T11:48:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.390+0000 7ffa7f7fe640 1 --2- 172.21.15.116:0/2727237971 >> v2:172.21.15.143:3300/0 conn(0x7ffa800cec80 0x7ffa800d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.391+0000 7ffa7e7fc640 1 --2- 172.21.15.116:0/2727237971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa800ce740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009870 tx=0x7ffa7402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964f76449c7fa306 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.391+0000 7ffa7dffb640 1 -- 172.21.15.116:0/2727237971 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74031640 con 0x7ffa801617e0 2024-02-18T11:48:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.391+0000 7ffa7dffb640 1 -- 172.21.15.116:0/2727237971 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa740317a0 con 0x7ffa801617e0 2024-02-18T11:48:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.391+0000 7ffa7dffb640 1 -- 172.21.15.116:0/2727237971 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74031b30 con 0x7ffa801617e0 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 -- 172.21.15.116:0/2727237971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 msgr2=0x7ffa800ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 --2- 172.21.15.116:0/2727237971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa800ce740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009870 tx=0x7ffa7402b8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 -- 172.21.15.116:0/2727237971 shutdown_connections 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 --2- 172.21.15.116:0/2727237971 >> v2:172.21.15.143:3300/0 conn(0x7ffa800cec80 0x7ffa800d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 --2- 172.21.15.116:0/2727237971 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa800ce740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 -- 172.21.15.116:0/2727237971 >> 172.21.15.116:0/2727237971 conn(0x7ffa800b5480 msgr2=0x7ffa800b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 -- 172.21.15.116:0/2727237971 shutdown_connections 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.392+0000 7ffa8598c640 1 -- 172.21.15.116:0/2727237971 wait complete. 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.393+0000 7ffa8598c640 1 Processor -- start 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.393+0000 7ffa8598c640 1 -- start start 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa8598c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 0x7ffa800c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa8598c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa8015dbf0 0x7ffa8015ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa8598c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa80154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa8598c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa801f3b80 con 0x7ffa800cec80 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa8598c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80160190 con 0x7ffa8015dbf0 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa8598c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8015f580 con 0x7ffa801617e0 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 0x7ffa800c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 0x7ffa800c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36814/0 (socket says 172.21.15.116:36814) 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 -- 172.21.15.116:0/1535021451 learned_addr learned my addr 172.21.15.116:0/1535021451 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7e7fc640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa8015dbf0 0x7ffa8015ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7f7fe640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa80154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 -- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 msgr2=0x7ffa80154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa80154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 -- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa8015dbf0 msgr2=0x7ffa8015ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa8015dbf0 0x7ffa8015ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.394+0000 7ffa7effd640 1 -- 172.21.15.116:0/1535021451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa70009a70 con 0x7ffa800cec80 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.395+0000 7ffa7effd640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 0x7ffa800c9c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c4d0 tx=0x7ffa7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.395+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780074e0 con 0x7ffa800cec80 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.395+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa78007640 con 0x7ffa800cec80 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.395+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa740094d0 con 0x7ffa800cec80 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.395+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78004a20 con 0x7ffa800cec80 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.395+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8015e150 con 0x7ffa800cec80 2024-02-18T11:48:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.396+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa80067840 con 0x7ffa800cec80 2024-02-18T11:48:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.396+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7ffa78004b80 con 0x7ffa800cec80 2024-02-18T11:48:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.397+0000 7ffa67fff640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa6c06bc70 0x7ffa6c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.397+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ffa7808a2b0 con 0x7ffa800cec80 2024-02-18T11:48:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.397+0000 7ffa7e7fc640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa6c06bc70 0x7ffa6c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.397+0000 7ffa7e7fc640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa6c06bc70 0x7ffa6c06e130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffa74036000 tx=0x7ffa7403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.393 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.401+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa78054ce0 con 0x7ffa800cec80 2024-02-18T11:48:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.638+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ffa8015f970 con 0x7ffa800cec80 2024-02-18T11:48:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.640+0000 7ffa67fff640 1 -- 172.21.15.116:0/1535021451 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7ffa780583a0 con 0x7ffa800cec80 2024-02-18T11:48:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa6c06bc70 msgr2=0x7ffa6c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa6c06bc70 0x7ffa6c06e130 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffa74036000 tx=0x7ffa7403c040 comp rx=0 tx=0).stop 2024-02-18T11:48:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 msgr2=0x7ffa800c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 0x7ffa800c9c10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c4d0 tx=0x7ffa7800c8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 shutdown_connections 2024-02-18T11:48:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7ffa6c06bc70 0x7ffa6c06e130 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa801617e0 0x7ffa80154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa8015dbf0 0x7ffa8015ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 --2- 172.21.15.116:0/1535021451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa800cec80 0x7ffa800c9c10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 >> 172.21.15.116:0/1535021451 conn(0x7ffa800b5480 msgr2=0x7ffa800b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 shutdown_connections 2024-02-18T11:48:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.642+0000 7ffa8598c640 1 -- 172.21.15.116:0/1535021451 wait complete. 2024-02-18T11:48:42.645 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-02-18T11:48:42.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.754+0000 7fa8c8c2c640 1 Processor -- start 2024-02-18T11:48:42.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.754+0000 7fa8c8c2c640 1 -- start start 2024-02-18T11:48:42.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.754+0000 7fa8c8c2c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 0x7fa8c41664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.754+0000 7fa8c8c2c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40703c0 con 0x7fa8c415cb20 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.754+0000 7fa8c8c2c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40cd820 con 0x7fa8c40afc00 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.754+0000 7fa8c8c2c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40affe0 con 0x7fa8c40d0080 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8bbfff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa8c415cb20 0x7fa8c40cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55830/0 (socket says 172.21.15.116:55830) 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8bbfff640 1 -- 172.21.15.116:0/3373683315 learned_addr learned my addr 172.21.15.116:0/3373683315 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c2d9d640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 0x7fa8c41664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481558167 0 0) 0x7fa8c40703c0 con 0x7fa8c415cb20 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b0003520 con 0x7fa8c415cb20 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255656341 0 0) 0x7fa8b0003520 con 0x7fa8c415cb20 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 msgr2=0x7fa8c41664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 0x7fa8c41664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 >> v1:172.21.15.143:6789/0 conn(0x7fa8c40afc00 legacy=0x7fa8c406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40ccef0 con 0x7fa8c415cb20 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fa8ac001670 con 0x7fa8c415cb20 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 >> v1:172.21.15.116:6789/0 conn(0x7fa8c415cb20 legacy=0x7fa8c40cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8b00039a0 0x7fa8b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 0x7fa8b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40affe0 con 0x7fa8b0004d00 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b000bb40 con 0x7fa8b00039a0 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b000bd00 con 0x7fa8c40d0080 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c2d9d640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8c259c640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8b00039a0 0x7fa8b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.755+0000 7fa8bbfff640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 0x7fa8b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c2d9d640 1 -- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 msgr2=0x7fa8b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c2d9d640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c40d0080 0x7fa8b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.752 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c2d9d640 1 -- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8b00039a0 msgr2=0x7fa8b0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c2d9d640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8b00039a0 0x7fa8b0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c2d9d640 1 -- 172.21.15.116:0/3373683315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b0003520 con 0x7fa8b0004d00 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c2d9d640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8b0005180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40cac00 tx=0x7fa8b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261809884fd403af server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02cb90 con 0x7fa8b0004d00 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00f040 con 0x7fa8b0004d00 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.756+0000 7fa8c1d9b640 1 -- 172.21.15.116:0/3373683315 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc034440 con 0x7fa8b0004d00 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/3373683315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 msgr2=0x7fa8b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/3373683315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8b0005180 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40cac00 tx=0x7fa8b00057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c2d9d640 1 -- 172.21.15.116:0/3373683315 reap_dead start 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/3373683315 shutdown_connections 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/3373683315 >> 172.21.15.116:0/3373683315 conn(0x7fa8c40b2230 msgr2=0x7fa8c40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/3373683315 shutdown_connections 2024-02-18T11:48:42.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.757+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/3373683315 wait complete. 2024-02-18T11:48:42.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.758+0000 7fa8c8c2c640 1 Processor -- start 2024-02-18T11:48:42.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.758+0000 7fa8c8c2c640 1 -- start start 2024-02-18T11:48:42.756 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c8c2c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8c4156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c8c2c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8c41f1c80 0x7fa8c41f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c8c2c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c41f28c0 0x7fa8c40b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c8c2c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4156a40 con 0x7fa8b0004d00 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c8c2c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40c7870 con 0x7fa8c41f1c80 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c8c2c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40b42d0 con 0x7fa8c41f28c0 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8c4156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8c4156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36832/0 (socket says 172.21.15.116:36832) 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 -- 172.21.15.116:0/2560147301 learned_addr learned my addr 172.21.15.116:0/2560147301 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c2d9d640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c41f28c0 0x7fa8c40b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8bbfff640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8c41f1c80 0x7fa8c41f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.757 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 -- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c41f28c0 msgr2=0x7fa8c40b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c41f28c0 0x7fa8c40b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 -- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8c41f1c80 msgr2=0x7fa8c41f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8c41f1c80 0x7fa8c41f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8c259c640 1 -- 172.21.15.116:0/2560147301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40b36c0 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.759+0000 7fa8bbfff640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8c41f1c80 0x7fa8c41f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.760+0000 7fa8c259c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8c4156500 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400f820 tx=0x7fa8b400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.760+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40050d0 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.760+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4005230 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.760+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c40b39a0 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.760+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400d730 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.760+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c40b3f60 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.761+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c4074520 con 0x7fa8b0004d00 2024-02-18T11:48:42.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.761+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa8b400d890 con 0x7fa8b0004d00 2024-02-18T11:48:42.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.762+0000 7fa8baffd640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8a406bca0 0x7fa8a406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:42.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.762+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa8b408a4d0 con 0x7fa8b0004d00 2024-02-18T11:48:42.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.762+0000 7fa8bbfff640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8a406bca0 0x7fa8a406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:42.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.762+0000 7fa8bbfff640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8a406bca0 0x7fa8a406e160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0010e0 tx=0x7fa8ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:42.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:42.766+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8b4054e70 con 0x7fa8b0004d00 2024-02-18T11:48:42.998 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.005+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa8c406e410 con 0x7fa8b0004d00 2024-02-18T11:48:42.999 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.007+0000 7fa8baffd640 1 -- 172.21.15.116:0/2560147301 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa8b4058530 con 0x7fa8b0004d00 2024-02-18T11:48:43.002 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8a406bca0 msgr2=0x7fa8a406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8a406bca0 0x7fa8a406e160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0010e0 tx=0x7fa8ac009040 comp rx=0 tx=0).stop 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 msgr2=0x7fa8c4156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8c4156500 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400f820 tx=0x7fa8b400fce0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 shutdown_connections 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8a406bca0 0x7fa8a406e160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8c41f28c0 0x7fa8c40b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8c41f1c80 0x7fa8c41f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 --2- 172.21.15.116:0/2560147301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8b0004d00 0x7fa8c4156500 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 >> 172.21.15.116:0/2560147301 conn(0x7fa8c40b2230 msgr2=0x7fa8c406fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:43.003 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.009+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 shutdown_connections 2024-02-18T11:48:43.004 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.010+0000 7fa8c8c2c640 1 -- 172.21.15.116:0/2560147301 wait complete. 2024-02-18T11:48:43.014 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-02-18T11:48:43.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.123+0000 7f792c676640 1 Processor -- start 2024-02-18T11:48:43.115 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.123+0000 7f792c676640 1 -- start start 2024-02-18T11:48:43.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792c676640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f79240d0080 0x7f79241664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792c676640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79240703c0 con 0x7f792415cb20 2024-02-18T11:48:43.116 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792c676640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79240cd820 con 0x7f79240afc00 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792c676640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240affe0 con 0x7f79240d0080 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f7929c11640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f792415cb20 0x7f79240cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55836/0 (socket says 172.21.15.116:55836) 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f7929c11640 1 -- 172.21.15.116:0/459738754 learned_addr learned my addr 172.21.15.116:0/459738754 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792ac13640 1 --2- 172.21.15.116:0/459738754 >> v2:172.21.15.116:3300/0 conn(0x7f79240d0080 0x7f79241664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792ac13640 1 -- 172.21.15.116:0/459738754 >> v1:172.21.15.143:6789/0 conn(0x7f79240afc00 legacy=0x7f792406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.124+0000 7f792ac13640 1 -- 172.21.15.116:0/459738754 >> v1:172.21.15.116:6789/0 conn(0x7f792415cb20 legacy=0x7f79240cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.117 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.125+0000 7f792ac13640 1 -- 172.21.15.116:0/459738754 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240ccef0 con 0x7f79240d0080 2024-02-18T11:48:43.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.125+0000 7f792ac13640 1 --2- 172.21.15.116:0/459738754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240d0080 0x7f79241664d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f791c009d90 tx=0x7f791c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd9884b91a1c13f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.125+0000 7f7929410640 1 -- 172.21.15.116:0/459738754 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c02cb40 con 0x7f79240d0080 2024-02-18T11:48:43.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.125+0000 7f7929410640 1 -- 172.21.15.116:0/459738754 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c02cca0 con 0x7f79240d0080 2024-02-18T11:48:43.118 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.125+0000 7f7929410640 1 -- 172.21.15.116:0/459738754 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c034660 con 0x7f79240d0080 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 -- 172.21.15.116:0/459738754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240d0080 msgr2=0x7f79241664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 --2- 172.21.15.116:0/459738754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240d0080 0x7f79241664d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f791c009d90 tx=0x7f791c02bd90 comp rx=0 tx=0).stop 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 -- 172.21.15.116:0/459738754 shutdown_connections 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 --2- 172.21.15.116:0/459738754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240d0080 0x7f79241664d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 -- 172.21.15.116:0/459738754 >> 172.21.15.116:0/459738754 conn(0x7f79240b2230 msgr2=0x7f79240b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 -- 172.21.15.116:0/459738754 shutdown_connections 2024-02-18T11:48:43.119 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.126+0000 7f792c676640 1 -- 172.21.15.116:0/459738754 wait complete. 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.127+0000 7f792c676640 1 Processor -- start 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.127+0000 7f792c676640 1 -- start start 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792c676640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 0x7f79240b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792c676640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79240d0080 0x7f792414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792c676640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f792415cb20 0x7f79240c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792c676640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792415ac50 con 0x7f79240afc00 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792c676640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924159d80 con 0x7f79240d0080 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792c676640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7924159fd0 con 0x7f792415cb20 2024-02-18T11:48:43.121 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 0x7f79240b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 0x7f79240b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36848/0 (socket says 172.21.15.116:36848) 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 -- 172.21.15.116:0/1258656926 learned_addr learned my addr 172.21.15.116:0/1258656926 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f7929c11640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79240d0080 0x7f792414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 -- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f792415cb20 msgr2=0x7f79240c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f792415cb20 0x7f79240c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 -- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79240d0080 msgr2=0x7f792414d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79240d0080 0x7f792414d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 -- 172.21.15.116:0/1258656926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c009a70 con 0x7f79240afc00 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.128+0000 7f792a412640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 0x7f79240b3320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f792000b3f0 tx=0x7f792000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.129+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920011830 con 0x7f79240afc00 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.129+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920011990 con 0x7f79240afc00 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.129+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792415a2b0 con 0x7f79240afc00 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.129+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920011d40 con 0x7f79240afc00 2024-02-18T11:48:43.122 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.129+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792415a870 con 0x7f79240afc00 2024-02-18T11:48:43.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.130+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79240739f0 con 0x7f79240afc00 2024-02-18T11:48:43.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.130+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f792001a400 con 0x7f79240afc00 2024-02-18T11:48:43.123 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.130+0000 7f791affd640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f790806bc70 0x7f790806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.130+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f792008a300 con 0x7f79240afc00 2024-02-18T11:48:43.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.130+0000 7f7929c11640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f790806bc70 0x7f790806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.124 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.131+0000 7f7929c11640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f790806bc70 0x7f790806e130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f792406f6f0 tx=0x7f7914007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.127 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.135+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7920054ca0 con 0x7f79240afc00 2024-02-18T11:48:43.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.373+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f7924074520 con 0x7f79240afc00 2024-02-18T11:48:43.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.375+0000 7f791affd640 1 -- 172.21.15.116:0/1258656926 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7920058360 con 0x7f79240afc00 2024-02-18T11:48:43.369 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f790806bc70 msgr2=0x7f790806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f790806bc70 0x7f790806e130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f792406f6f0 tx=0x7f7914007e80 comp rx=0 tx=0).stop 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 msgr2=0x7f79240b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 0x7f79240b3320 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f792000b3f0 tx=0x7f792000b8b0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 shutdown_connections 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f790806bc70 0x7f790806e130 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f792415cb20 0x7f79240c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79240d0080 0x7f792414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 --2- 172.21.15.116:0/1258656926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79240afc00 0x7f79240b3320 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 >> 172.21.15.116:0/1258656926 conn(0x7f79240b2230 msgr2=0x7f792406e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 shutdown_connections 2024-02-18T11:48:43.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.377+0000 7f792c676640 1 -- 172.21.15.116:0/1258656926 wait complete. 2024-02-18T11:48:43.380 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-02-18T11:48:43.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.488+0000 7f8959d77640 1 Processor -- start 2024-02-18T11:48:43.481 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.488+0000 7f8959d77640 1 -- start start 2024-02-18T11:48:43.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8959d77640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89540b5170 0x7f89540b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.482 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8959d77640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f895406bf50 0x7f895406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8959d77640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540b5b30 con 0x7f895415db40 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8959d77640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8954070f50 con 0x7f895406bf50 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8959d77640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8954071230 con 0x7f89540b5170 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8953fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f895406bf50 0x7f895406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8953fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f895406bf50 0x7f895406c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36852/0 (socket says 172.21.15.116:36852) 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.489+0000 7f8953fff640 1 -- 172.21.15.116:0/4008357798 learned_addr learned my addr 172.21.15.116:0/4008357798 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f8952ffd640 1 --2- 172.21.15.116:0/4008357798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89540b5170 0x7f89540b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f8953fff640 1 -- 172.21.15.116:0/4008357798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89540b5170 msgr2=0x7f89540b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f8953fff640 1 --2- 172.21.15.116:0/4008357798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89540b5170 0x7f89540b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f8953fff640 1 -- 172.21.15.116:0/4008357798 >> v1:172.21.15.143:6789/0 conn(0x7f895415db40 legacy=0x7f89540b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f8953fff640 1 -- 172.21.15.116:0/4008357798 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954071510 con 0x7f895406bf50 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f8953fff640 1 --2- 172.21.15.116:0/4008357798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895406c3e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f894c009e50 tx=0x7f894c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f457b2fd81f5428c server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.490+0000 7f89527fc640 1 -- 172.21.15.116:0/4008357798 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c02cb00 con 0x7f895406bf50 2024-02-18T11:48:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f89527fc640 1 -- 172.21.15.116:0/4008357798 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c02cc60 con 0x7f895406bf50 2024-02-18T11:48:43.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f89527fc640 1 -- 172.21.15.116:0/4008357798 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0343f0 con 0x7f895406bf50 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 -- 172.21.15.116:0/4008357798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 msgr2=0x7f895406c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 --2- 172.21.15.116:0/4008357798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895406c3e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f894c009e50 tx=0x7f894c02bf60 comp rx=0 tx=0).stop 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 -- 172.21.15.116:0/4008357798 shutdown_connections 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 --2- 172.21.15.116:0/4008357798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895406c3e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 --2- 172.21.15.116:0/4008357798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f89540b5170 0x7f89540b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 -- 172.21.15.116:0/4008357798 >> 172.21.15.116:0/4008357798 conn(0x7f895406f460 msgr2=0x7f895406f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 -- 172.21.15.116:0/4008357798 shutdown_connections 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.491+0000 7f8959d77640 1 -- 172.21.15.116:0/4008357798 wait complete. 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.492+0000 7f8959d77640 1 Processor -- start 2024-02-18T11:48:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.492+0000 7f8959d77640 1 -- start start 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8959d77640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895414c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8959d77640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540b5170 0x7f8954158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8959d77640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f895415db40 0x7f8954159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8959d77640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954157450 con 0x7f895406bf50 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8959d77640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541563b0 con 0x7f89540b5170 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8959d77640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8954156690 con 0x7f895415db40 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f89537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895414c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.487 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f89537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895414c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36862/0 (socket says 172.21.15.116:36862) 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f89537fe640 1 -- 172.21.15.116:0/2451409834 learned_addr learned my addr 172.21.15.116:0/2451409834 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8953fff640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f895415db40 0x7f8954159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.493+0000 7f8952ffd640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540b5170 0x7f8954158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f89537fe640 1 -- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f895415db40 msgr2=0x7f8954159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f89537fe640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f895415db40 0x7f8954159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f89537fe640 1 -- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540b5170 msgr2=0x7f8954158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f89537fe640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540b5170 0x7f8954158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f89537fe640 1 -- 172.21.15.116:0/2451409834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009b30 con 0x7f895406bf50 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f89537fe640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895414c4f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f894400c3c0 tx=0x7f894400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440052d0 con 0x7f895406bf50 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944005450 con 0x7f895406bf50 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8954156970 con 0x7f895406bf50 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944007260 con 0x7f895406bf50 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.494+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8954156f30 con 0x7f895406bf50 2024-02-18T11:48:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.495+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89540b6170 con 0x7f895406bf50 2024-02-18T11:48:43.489 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.495+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f89440073c0 con 0x7f895406bf50 2024-02-18T11:48:43.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.496+0000 7f893bfff640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f894006bca0 0x7f894006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.496+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f89440898c0 con 0x7f895406bf50 2024-02-18T11:48:43.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.496+0000 7f8952ffd640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f894006bca0 0x7f894006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.490 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.496+0000 7f8952ffd640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f894006bca0 0x7f894006e160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89540b15b0 tx=0x7f8948008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.492 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.500+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8944054310 con 0x7f895406bf50 2024-02-18T11:48:43.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.735+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8954157150 con 0x7f895406bf50 2024-02-18T11:48:43.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.737+0000 7f893bfff640 1 -- 172.21.15.116:0/2451409834 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f89440579d0 con 0x7f895406bf50 2024-02-18T11:48:43.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f894006bca0 msgr2=0x7f894006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f894006bca0 0x7f894006e160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89540b15b0 tx=0x7f8948008040 comp rx=0 tx=0).stop 2024-02-18T11:48:43.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 msgr2=0x7f895414c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895414c4f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f894400c3c0 tx=0x7f894400c880 comp rx=0 tx=0).stop 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 shutdown_connections 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f894006bca0 0x7f894006e160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f895415db40 0x7f8954159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540b5170 0x7f8954158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 --2- 172.21.15.116:0/2451409834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f895406bf50 0x7f895414c4f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 >> 172.21.15.116:0/2451409834 conn(0x7f895406f460 msgr2=0x7f895415b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 shutdown_connections 2024-02-18T11:48:43.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.739+0000 7f8959d77640 1 -- 172.21.15.116:0/2451409834 wait complete. 2024-02-18T11:48:43.742 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:43.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.850+0000 7fd92cb2c640 1 Processor -- start 2024-02-18T11:48:43.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.850+0000 7fd92cb2c640 1 -- start start 2024-02-18T11:48:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd92cb2c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd928158260 0x7fd92815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd92cb2c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9280b37d0 0x7fd92806ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd92cb2c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9280b3e10 con 0x7fd92806da20 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd92cb2c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9280b5360 con 0x7fd9280b37d0 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd92cb2c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9281615c0 con 0x7fd928158260 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9280b37d0 0x7fd92806ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9280b37d0 0x7fd92806ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36866/0 (socket says 172.21.15.116:36866) 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 -- 172.21.15.116:0/2737570106 learned_addr learned my addr 172.21.15.116:0/2737570106 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd92659c640 1 --2- 172.21.15.116:0/2737570106 >> v2:172.21.15.143:3300/0 conn(0x7fd928158260 0x7fd92815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 -- 172.21.15.116:0/2737570106 >> v2:172.21.15.143:3300/0 conn(0x7fd928158260 msgr2=0x7fd92815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 --2- 172.21.15.116:0/2737570106 >> v2:172.21.15.143:3300/0 conn(0x7fd928158260 0x7fd92815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 -- 172.21.15.116:0/2737570106 >> v1:172.21.15.116:6789/0 conn(0x7fd92806da20 legacy=0x7fd9280b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.851+0000 7fd926d9d640 1 -- 172.21.15.116:0/2737570106 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9281618a0 con 0x7fd9280b37d0 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.852+0000 7fd926d9d640 1 --2- 172.21.15.116:0/2737570106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9280b37d0 0x7fd92806ada0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd92809e0b0 tx=0x7fd92002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b895a3fbf1fb97 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.852+0000 7fd92559a640 1 -- 172.21.15.116:0/2737570106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92002cb20 con 0x7fd9280b37d0 2024-02-18T11:48:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.852+0000 7fd92559a640 1 -- 172.21.15.116:0/2737570106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92002cc80 con 0x7fd9280b37d0 2024-02-18T11:48:43.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.852+0000 7fd92559a640 1 -- 172.21.15.116:0/2737570106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9200343f0 con 0x7fd9280b37d0 2024-02-18T11:48:43.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 -- 172.21.15.116:0/2737570106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9280b37d0 msgr2=0x7fd92806ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/2737570106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9280b37d0 0x7fd92806ada0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd92809e0b0 tx=0x7fd92002bf80 comp rx=0 tx=0).stop 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 -- 172.21.15.116:0/2737570106 shutdown_connections 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/2737570106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9280b37d0 0x7fd92806ada0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/2737570106 >> v2:172.21.15.143:3300/0 conn(0x7fd928158260 0x7fd92815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 -- 172.21.15.116:0/2737570106 >> 172.21.15.116:0/2737570106 conn(0x7fd9280b23d0 msgr2=0x7fd9280cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 -- 172.21.15.116:0/2737570106 shutdown_connections 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.853+0000 7fd92cb2c640 1 -- 172.21.15.116:0/2737570106 wait complete. 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.854+0000 7fd92cb2c640 1 Processor -- start 2024-02-18T11:48:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.854+0000 7fd92cb2c640 1 -- start start 2024-02-18T11:48:43.849 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92cb2c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd92806da20 0x7fd92814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92cb2c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9280b37d0 0x7fd9280acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92cb2c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 0x7fd9280ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92cb2c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9280cc9d0 con 0x7fd928158260 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92cb2c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92814f6f0 con 0x7fd92806da20 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92cb2c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92814f9d0 con 0x7fd9280b37d0 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 0x7fd9280ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 0x7fd9280ad2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36878/0 (socket says 172.21.15.116:36878) 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 -- 172.21.15.116:0/949164864 learned_addr learned my addr 172.21.15.116:0/949164864 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd92659c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd92806da20 0x7fd92814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd925d9b640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9280b37d0 0x7fd9280acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 -- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9280b37d0 msgr2=0x7fd9280acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9280b37d0 0x7fd9280acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 -- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd92806da20 msgr2=0x7fd92814c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:43.850 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd92806da20 0x7fd92814c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd926d9d640 1 -- 172.21.15.116:0/949164864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd920009b30 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.855+0000 7fd925d9b640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9280b37d0 0x7fd9280acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.856+0000 7fd926d9d640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 0x7fd9280ad2d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd920036040 tx=0x7fd920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.856+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920007e70 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.856+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920009510 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.856+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92814fcb0 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.856+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92003e690 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.856+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92806f5b0 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.857+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9280ccdf0 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.857+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fd92003e7f0 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.857+0000 7fd916ffd640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd90806bc70 0x7fd90806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.858+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd9200ad0a0 con 0x7fd928158260 2024-02-18T11:48:43.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.858+0000 7fd92659c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd90806bc70 0x7fd90806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:43.852 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.858+0000 7fd92659c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd90806bc70 0x7fd90806e130 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd92814c7e0 tx=0x7fd918008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:43.854 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:43.862+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9280ccdf0 con 0x7fd928158260 2024-02-18T11:48:44.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.032+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9280ccdf0 con 0x7fd928158260 2024-02-18T11:48:44.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.032+0000 7fd916ffd640 1 -- 172.21.15.116:0/949164864 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13176 (secure 0 0 0) 0x7fd9280ccdf0 con 0x7fd928158260 2024-02-18T11:48:44.025 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:44.025 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:41.176253+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd90806bc70 msgr2=0x7fd90806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd90806bc70 0x7fd90806e130 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd92814c7e0 tx=0x7fd918008040 comp rx=0 tx=0).stop 2024-02-18T11:48:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 msgr2=0x7fd9280ad2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 0x7fd9280ad2d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd920036040 tx=0x7fd920004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 shutdown_connections 2024-02-18T11:48:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fd90806bc70 0x7fd90806e130 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd928158260 0x7fd9280ad2d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd9280b37d0 0x7fd9280acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 --2- 172.21.15.116:0/949164864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd92806da20 0x7fd92814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 >> 172.21.15.116:0/949164864 conn(0x7fd9280b23d0 msgr2=0x7fd92806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 shutdown_connections 2024-02-18T11:48:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.034+0000 7fd92cb2c640 1 -- 172.21.15.116:0/949164864 wait complete. 2024-02-18T11:48:44.038 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T11:48:44.038 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T11:48:44.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c86ceb640 1 Processor -- start 2024-02-18T11:48:44.142 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c86ceb640 1 -- start start 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c86ceb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c800cee50 0x7f3c80168b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c86ceb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c800d1ee0 con 0x7f3c8015dbf0 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c86ceb640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c80161dc0 con 0x7f3c801617e0 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c86ceb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c801620a0 con 0x7f3c800cee50 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.150+0000 7f3c85288640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c800cee50 0x7f3c80168b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c84a87640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3c8015dbf0 0x7f3c800b9370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55854/0 (socket says 172.21.15.116:55854) 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c85288640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c800cee50 0x7f3c80168b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36884/0 (socket says 172.21.15.116:36884) 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c84a87640 1 -- 172.21.15.116:0/204866626 learned_addr learned my addr 172.21.15.116:0/204866626 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:44.143 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c85288640 1 -- 172.21.15.116:0/204866626 >> v1:172.21.15.143:6789/0 conn(0x7f3c801617e0 legacy=0x7f3c800ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c85288640 1 -- 172.21.15.116:0/204866626 >> v1:172.21.15.116:6789/0 conn(0x7f3c8015dbf0 legacy=0x7f3c800b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c85288640 1 -- 172.21.15.116:0/204866626 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c80162380 con 0x7f3c800cee50 2024-02-18T11:48:44.144 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.151+0000 7f3c7f7fe640 1 -- 172.21.15.116:0/204866626 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868085870 0 0) 0x7f3c800d1ee0 con 0x7f3c8015dbf0 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c85288640 1 --2- 172.21.15.116:0/204866626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c80168b10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009d90 tx=0x7f3c7802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed902d7172f8a80 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c7f7fe640 1 -- 172.21.15.116:0/204866626 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7802cc60 con 0x7f3c800cee50 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c7f7fe640 1 -- 172.21.15.116:0/204866626 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7802cdc0 con 0x7f3c800cee50 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c7f7fe640 1 -- 172.21.15.116:0/204866626 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78034660 con 0x7f3c800cee50 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 -- 172.21.15.116:0/204866626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 msgr2=0x7f3c80168b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/204866626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c80168b10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009d90 tx=0x7f3c7802beb0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 -- 172.21.15.116:0/204866626 shutdown_connections 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/204866626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c80168b10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 -- 172.21.15.116:0/204866626 >> 172.21.15.116:0/204866626 conn(0x7f3c800b5480 msgr2=0x7f3c800b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:44.145 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 -- 172.21.15.116:0/204866626 shutdown_connections 2024-02-18T11:48:44.146 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.152+0000 7f3c86ceb640 1 -- 172.21.15.116:0/204866626 wait complete. 2024-02-18T11:48:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.153+0000 7f3c86ceb640 1 Processor -- start 2024-02-18T11:48:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.153+0000 7f3c86ceb640 1 -- start start 2024-02-18T11:48:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c86ceb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c8015cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c86ceb640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3c8015dbf0 0x7f3c8015f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c86ceb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c801617e0 0x7f3c8014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c86ceb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8014e480 con 0x7f3c800cee50 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c86ceb640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801f7e60 con 0x7f3c801617e0 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c86ceb640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801f6f70 con 0x7f3c8015dbf0 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c84a87640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c8015cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c84a87640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c8015cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36886/0 (socket says 172.21.15.116:36886) 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.154+0000 7f3c84a87640 1 -- 172.21.15.116:0/2555793414 learned_addr learned my addr 172.21.15.116:0/2555793414 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c84a87640 1 -- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3c8015dbf0 msgr2=0x7f3c8015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c84a87640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3c8015dbf0 0x7f3c8015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c84a87640 1 -- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c801617e0 msgr2=0x7f3c8014df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c85288640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c801617e0 0x7f3c8014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c84a87640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c801617e0 0x7f3c8014df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c84a87640 1 -- 172.21.15.116:0/2555793414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c78009a70 con 0x7f3c800cee50 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c84a87640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c8015cc40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c4d0 tx=0x7f3c7400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c740074e0 con 0x7f3c800cee50 2024-02-18T11:48:44.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74007640 con 0x7f3c800cee50 2024-02-18T11:48:44.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004a20 con 0x7f3c800cee50 2024-02-18T11:48:44.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8015e150 con 0x7f3c800cee50 2024-02-18T11:48:44.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.155+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c801f7530 con 0x7f3c800cee50 2024-02-18T11:48:44.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.156+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c80067840 con 0x7f3c800cee50 2024-02-18T11:48:44.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.156+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3c74004b80 con 0x7f3c800cee50 2024-02-18T11:48:44.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.157+0000 7f3c7d7fa640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3c6406bc70 0x7f3c6406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.157+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3c74089a60 con 0x7f3c800cee50 2024-02-18T11:48:44.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.157+0000 7f3c7ffff640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3c6406bc70 0x7f3c6406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.152 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.157+0000 7f3c7ffff640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3c6406bc70 0x7f3c6406e130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c000f60 tx=0x7f3c6c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.161+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c74054490 con 0x7f3c800cee50 2024-02-18T11:48:44.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.335+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c80074520 con 0x7f3c800cee50 2024-02-18T11:48:44.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.336+0000 7f3c7d7fa640 1 -- 172.21.15.116:0/2555793414 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c74057b50 con 0x7f3c800cee50 2024-02-18T11:48:44.328 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T11:48:44.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3c6406bc70 msgr2=0x7f3c6406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3c6406bc70 0x7f3c6406e130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c000f60 tx=0x7f3c6c0079b0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 msgr2=0x7f3c8015cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c8015cc40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400c4d0 tx=0x7f3c7400c990 comp rx=0 tx=0).stop 2024-02-18T11:48:44.330 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 shutdown_connections 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3c6406bc70 0x7f3c6406e130 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c801617e0 0x7f3c8014df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3c8015dbf0 0x7f3c8015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 --2- 172.21.15.116:0/2555793414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800cee50 0x7f3c8015cc40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 >> 172.21.15.116:0/2555793414 conn(0x7f3c800b5480 msgr2=0x7f3c8006c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 shutdown_connections 2024-02-18T11:48:44.331 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.338+0000 7f3c86ceb640 1 -- 172.21.15.116:0/2555793414 wait complete. 2024-02-18T11:48:44.341 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-18T11:48:44.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.450+0000 7f748c876640 1 Processor -- start 2024-02-18T11:48:44.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.450+0000 7f748c876640 1 -- start start 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748c876640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f74880cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748c876640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74880cd1f0 0x7f748806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748c876640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f748815ac40 0x7f748806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748c876640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880b02e0 con 0x7f748815ac40 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748c876640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880af6d0 con 0x7f74880cd1f0 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748c876640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880af9b0 con 0x7f748815a860 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f748815ac40 0x7f748806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f748815ac40 0x7f748806e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36892/0 (socket says 172.21.15.116:36892) 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 -- 172.21.15.116:0/1102730242 learned_addr learned my addr 172.21.15.116:0/1102730242 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f747ffff640 1 --2- 172.21.15.116:0/1102730242 >> v2:172.21.15.143:3300/0 conn(0x7f74880cd1f0 0x7f748806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f748659c640 1 --2- 172.21.15.116:0/1102730242 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f74880cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 -- 172.21.15.116:0/1102730242 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 msgr2=0x7f74880cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:44.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 --2- 172.21.15.116:0/1102730242 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f74880cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 -- 172.21.15.116:0/1102730242 >> v2:172.21.15.143:3300/0 conn(0x7f74880cd1f0 msgr2=0x7f748806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 --2- 172.21.15.116:0/1102730242 >> v2:172.21.15.143:3300/0 conn(0x7f74880cd1f0 0x7f748806e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.451+0000 7f7486d9d640 1 -- 172.21.15.116:0/1102730242 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880afc90 con 0x7f748815ac40 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f7486d9d640 1 --2- 172.21.15.116:0/1102730242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748806e850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f748806ef50 tx=0x7f748002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0d2c38198a594 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f7485d9b640 1 -- 172.21.15.116:0/1102730242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748002ca70 con 0x7f748815ac40 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f7485d9b640 1 -- 172.21.15.116:0/1102730242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748002cbd0 con 0x7f748815ac40 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f7485d9b640 1 -- 172.21.15.116:0/1102730242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74800343f0 con 0x7f748815ac40 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f748c876640 1 -- 172.21.15.116:0/1102730242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 msgr2=0x7f748806e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f748c876640 1 --2- 172.21.15.116:0/1102730242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748806e850 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f748806ef50 tx=0x7f748002bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f748c876640 1 -- 172.21.15.116:0/1102730242 shutdown_connections 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.452+0000 7f748c876640 1 --2- 172.21.15.116:0/1102730242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748806e850 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.453+0000 7f748c876640 1 --2- 172.21.15.116:0/1102730242 >> v2:172.21.15.143:3300/0 conn(0x7f74880cd1f0 0x7f748806e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.453+0000 7f748c876640 1 --2- 172.21.15.116:0/1102730242 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f74880cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.453+0000 7f748c876640 1 -- 172.21.15.116:0/1102730242 >> 172.21.15.116:0/1102730242 conn(0x7f7488066f20 msgr2=0x7f748806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:44.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.453+0000 7f748c876640 1 -- 172.21.15.116:0/1102730242 shutdown_connections 2024-02-18T11:48:44.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.453+0000 7f748c876640 1 -- 172.21.15.116:0/1102730242 wait complete. 2024-02-18T11:48:44.449 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.454+0000 7f748c876640 1 Processor -- start 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.454+0000 7f748c876640 1 -- start start 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74880cd1f0 0x7f7488153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f7488158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748814fa40 con 0x7f748815ac40 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748814edc0 con 0x7f74880cd1f0 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748814f0a0 con 0x7f748815a860 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748814fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36904/0 (socket says 172.21.15.116:36904) 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 -- 172.21.15.116:0/2624530942 learned_addr learned my addr 172.21.15.116:0/2624530942 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748659c640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74880cd1f0 0x7f7488153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f747ffff640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f7488158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 -- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 msgr2=0x7f7488158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f7488158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 -- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74880cd1f0 msgr2=0x7f7488153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74880cd1f0 0x7f7488153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 -- 172.21.15.116:0/2624530942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480009b30 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f747ffff640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f7488158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f7486d9d640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748814fe60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7480009f40 tx=0x7f7480004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480009510 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748003e450 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748814f380 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748003e7e0 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.455+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7488153dc0 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.456+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748814fb80 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.457+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f748003e940 con 0x7f748815ac40 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.457+0000 7f747effd640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746806bc70 0x7f746806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.457+0000 7f748659c640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746806bc70 0x7f746806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.451 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.457+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f74800ad320 con 0x7f748815ac40 2024-02-18T11:48:44.452 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.458+0000 7f748659c640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746806bc70 0x7f746806e130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7478006fd0 tx=0x7f7478008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.462+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7480077d50 con 0x7f748815ac40 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.633+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7488074520 con 0x7f748815ac40 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.634+0000 7f747effd640 1 -- 172.21.15.116:0/2624530942 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f748007b410 con 0x7f748815ac40 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stdout: { 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": ".mgr" 2024-02-18T11:48:44.626 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-02-18T11:48:44.627 INFO:teuthology.orchestra.run.smithi116.stdout:] 2024-02-18T11:48:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746806bc70 msgr2=0x7f746806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746806bc70 0x7f746806e130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7478006fd0 tx=0x7f7478008040 comp rx=0 tx=0).stop 2024-02-18T11:48:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 msgr2=0x7f748814fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748814fe60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7480009f40 tx=0x7f7480004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:44.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 shutdown_connections 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f746806bc70 0x7f746806e130 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f748815ac40 0x7f748814fe60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748815a860 0x7f7488158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 --2- 172.21.15.116:0/2624530942 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74880cd1f0 0x7f7488153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 >> 172.21.15.116:0/2624530942 conn(0x7f7488066f20 msgr2=0x7f748815b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 shutdown_connections 2024-02-18T11:48:44.629 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.636+0000 7f748c876640 1 -- 172.21.15.116:0/2624530942 wait complete. 2024-02-18T11:48:44.639 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-18T11:48:44.639 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-18T11:48:44.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.750+0000 7faa643bc640 1 Processor -- start 2024-02-18T11:48:44.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.750+0000 7faa643bc640 1 -- start start 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa643bc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa5c15dbf0 0x7faa5c1617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa643bc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c0d1ee0 con 0x7faa5c0b9370 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa643bc640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c161dc0 con 0x7faa5c0cee50 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa643bc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c1620a0 con 0x7faa5c15dbf0 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa62158640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa5c15dbf0 0x7faa5c1617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa62158640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa5c15dbf0 0x7faa5c1617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36908/0 (socket says 172.21.15.116:36908) 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa62158640 1 -- 172.21.15.116:0/3123659186 learned_addr learned my addr 172.21.15.116:0/3123659186 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.751+0000 7faa61957640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7faa5c0b9370 0x7faa5c0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55858/0 (socket says 172.21.15.116:55858) 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.752+0000 7faa62158640 1 -- 172.21.15.116:0/3123659186 >> v1:172.21.15.143:6789/0 conn(0x7faa5c0cee50 legacy=0x7faa5c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.752+0000 7faa62158640 1 -- 172.21.15.116:0/3123659186 >> v1:172.21.15.116:6789/0 conn(0x7faa5c0b9370 legacy=0x7faa5c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.752+0000 7faa62158640 1 -- 172.21.15.116:0/3123659186 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c162380 con 0x7faa5c15dbf0 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.752+0000 7faa62158640 1 --2- 172.21.15.116:0/3123659186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c15dbf0 0x7faa5c1617e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194bfef13370b5c server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.752+0000 7faa61156640 1 -- 172.21.15.116:0/3123659186 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5802f840 con 0x7faa5c15dbf0 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa61156640 1 -- 172.21.15.116:0/3123659186 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5802f9a0 con 0x7faa5c15dbf0 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa61156640 1 -- 172.21.15.116:0/3123659186 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5802fd30 con 0x7faa5c15dbf0 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 -- 172.21.15.116:0/3123659186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c15dbf0 msgr2=0x7faa5c1617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 --2- 172.21.15.116:0/3123659186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c15dbf0 0x7faa5c1617e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faa58009870 tx=0x7faa5802ba20 comp rx=0 tx=0).stop 2024-02-18T11:48:44.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 -- 172.21.15.116:0/3123659186 shutdown_connections 2024-02-18T11:48:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 --2- 172.21.15.116:0/3123659186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c15dbf0 0x7faa5c1617e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 -- 172.21.15.116:0/3123659186 >> 172.21.15.116:0/3123659186 conn(0x7faa5c0b5480 msgr2=0x7faa5c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 -- 172.21.15.116:0/3123659186 shutdown_connections 2024-02-18T11:48:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.753+0000 7faa643bc640 1 -- 172.21.15.116:0/3123659186 wait complete. 2024-02-18T11:48:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.754+0000 7faa643bc640 1 Processor -- start 2024-02-18T11:48:44.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.754+0000 7faa643bc640 1 -- start start 2024-02-18T11:48:44.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa643bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 0x7faa5c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa643bc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5c0cee50 0x7faa5c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa643bc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa5c15dbf0 0x7faa5c160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa643bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c153a70 con 0x7faa5c0b9370 2024-02-18T11:48:44.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa643bc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c153d50 con 0x7faa5c15dbf0 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa643bc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c154030 con 0x7faa5c0cee50 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 0x7faa5c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 0x7faa5c154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36918/0 (socket says 172.21.15.116:36918) 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 -- 172.21.15.116:0/1284555910 learned_addr learned my addr 172.21.15.116:0/1284555910 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62959640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa5c15dbf0 0x7faa5c160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa61957640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5c0cee50 0x7faa5c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 -- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5c0cee50 msgr2=0x7faa5c152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5c0cee50 0x7faa5c152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 -- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa5c15dbf0 msgr2=0x7faa5c160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa5c15dbf0 0x7faa5c160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 -- 172.21.15.116:0/1284555910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa580094d0 con 0x7faa5c0b9370 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.755+0000 7faa62158640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 0x7faa5c154b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c00 tx=0x7faa58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.756+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5802f810 con 0x7faa5c0b9370 2024-02-18T11:48:44.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.756+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58041400 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.756+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c15e150 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.756+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580417d0 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.756+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c1545f0 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.757+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c067840 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.757+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7faa58041930 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.757+0000 7faa52ffd640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa4006bca0 0x7faa4006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.757+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7faa5803c070 con 0x7faa5c0b9370 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.757+0000 7faa61957640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa4006bca0 0x7faa4006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:44.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.758+0000 7faa61957640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa4006bca0 0x7faa4006e160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0cebd0 tx=0x7faa4c007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:44.754 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.762+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa58077e50 con 0x7faa5c0b9370 2024-02-18T11:48:44.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:44.934+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faa5c1547e0 con 0x7faa5c0b9370 2024-02-18T11:48:45.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.280+0000 7faa52ffd640 1 -- 172.21.15.116:0/1284555910 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7faa5807b510 con 0x7faa5c0b9370 2024-02-18T11:48:45.273 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa4006bca0 msgr2=0x7faa4006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa4006bca0 0x7faa4006e160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0cebd0 tx=0x7faa4c007c80 comp rx=0 tx=0).stop 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 msgr2=0x7faa5c154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 0x7faa5c154b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faa58000c00 tx=0x7faa58004730 comp rx=0 tx=0).stop 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 shutdown_connections 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7faa4006bca0 0x7faa4006e160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa5c15dbf0 0x7faa5c160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faa5c0cee50 0x7faa5c152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 --2- 172.21.15.116:0/1284555910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa5c0b9370 0x7faa5c154b10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 >> 172.21.15.116:0/1284555910 conn(0x7faa5c0b5480 msgr2=0x7faa5c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 shutdown_connections 2024-02-18T11:48:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.282+0000 7faa643bc640 1 -- 172.21.15.116:0/1284555910 wait complete. 2024-02-18T11:48:45.290 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-18T11:48:45.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.404+0000 7f20fd823640 1 Processor -- start 2024-02-18T11:48:45.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.404+0000 7f20fd823640 1 -- start start 2024-02-18T11:48:45.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20fd823640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f81617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20fd823640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20f80b9370 0x7f20f80ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20fd823640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20f80cec80 0x7f20f806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20fd823640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80cf110 con 0x7f20f80b9370 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20fd823640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80d1ee0 con 0x7f20f80cec80 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20fd823640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8161dc0 con 0x7f20f815dbf0 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20f80b9370 0x7f20f80ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20f80b9370 0x7f20f80ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36928/0 (socket says 172.21.15.116:36928) 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 -- 172.21.15.116:0/2496072894 learned_addr learned my addr 172.21.15.116:0/2496072894 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f77fe640 1 --2- 172.21.15.116:0/2496072894 >> v2:172.21.15.143:3300/0 conn(0x7f20f80cec80 0x7f20f806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f6ffd640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f81617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 -- 172.21.15.116:0/2496072894 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 msgr2=0x7f20f81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f81617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 -- 172.21.15.116:0/2496072894 >> v2:172.21.15.143:3300/0 conn(0x7f20f80cec80 msgr2=0x7f20f806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:45.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 --2- 172.21.15.116:0/2496072894 >> v2:172.21.15.143:3300/0 conn(0x7f20f80cec80 0x7f20f806c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f67fc640 1 -- 172.21.15.116:0/2496072894 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f81620a0 con 0x7f20f80b9370 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f77fe640 1 --2- 172.21.15.116:0/2496072894 >> v2:172.21.15.143:3300/0 conn(0x7f20f80cec80 0x7f20f806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.405+0000 7f20f6ffd640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.406+0000 7f20f67fc640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80ce740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f20f808c170 tx=0x7f20ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c6bcc1ea29752 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.406+0000 7f20f5ffb640 1 -- 172.21.15.116:0/2496072894 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec031870 con 0x7f20f80b9370 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.406+0000 7f20f5ffb640 1 -- 172.21.15.116:0/2496072894 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec0319d0 con 0x7f20f80b9370 2024-02-18T11:48:45.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.406+0000 7f20f5ffb640 1 -- 172.21.15.116:0/2496072894 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec031d60 con 0x7f20f80b9370 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 -- 172.21.15.116:0/2496072894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 msgr2=0x7f20f80ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80ce740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f20f808c170 tx=0x7f20ec02b910 comp rx=0 tx=0).stop 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 -- 172.21.15.116:0/2496072894 shutdown_connections 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 --2- 172.21.15.116:0/2496072894 >> v2:172.21.15.143:3300/0 conn(0x7f20f80cec80 0x7f20f806c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80ce740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 --2- 172.21.15.116:0/2496072894 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f81617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 -- 172.21.15.116:0/2496072894 >> 172.21.15.116:0/2496072894 conn(0x7f20f80b5480 msgr2=0x7f20f80b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 -- 172.21.15.116:0/2496072894 shutdown_connections 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.407+0000 7f20fd823640 1 -- 172.21.15.116:0/2496072894 wait complete. 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.408+0000 7f20fd823640 1 Processor -- start 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.408+0000 7f20fd823640 1 -- start start 2024-02-18T11:48:45.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20fd823640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20fd823640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f80cec80 0x7f20f815f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20fd823640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f8151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20fd823640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81600e0 con 0x7f20f80b9370 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20fd823640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81603c0 con 0x7f20f80cec80 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20fd823640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81606a0 con 0x7f20f815dbf0 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36934/0 (socket says 172.21.15.116:36934) 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 -- 172.21.15.116:0/2584440450 learned_addr learned my addr 172.21.15.116:0/2584440450 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f77fe640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f8151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f67fc640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f80cec80 0x7f20f815f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 -- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 msgr2=0x7f20f8151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f8151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 -- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f80cec80 msgr2=0x7f20f815f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f80cec80 0x7f20f815f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.409+0000 7f20f6ffd640 1 -- 172.21.15.116:0/2584440450 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0094d0 con 0x7f20f80b9370 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.410+0000 7f20f6ffd640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80b7b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c7f0 tx=0x7f20e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.410+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007820 con 0x7f20f80b9370 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.410+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8007980 con 0x7f20f80b9370 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.410+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8007cf0 con 0x7f20f80b9370 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.410+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f815e150 con 0x7f20f80b9370 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.410+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f8160c60 con 0x7f20f80b9370 2024-02-18T11:48:45.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.411+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f8067840 con 0x7f20f80b9370 2024-02-18T11:48:45.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.411+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f20e80113f0 con 0x7f20f80b9370 2024-02-18T11:48:45.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.412+0000 7f20dffff640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f20e406bc70 0x7f20e406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:45.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.412+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f20e808a8e0 con 0x7f20f80b9370 2024-02-18T11:48:45.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.412+0000 7f20f67fc640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f20e406bc70 0x7f20e406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:45.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.412+0000 7f20f67fc640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f20e406bc70 0x7f20e406e130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20ec036000 tx=0x7f20ec03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:45.409 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.416+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20e8055250 con 0x7f20f80b9370 2024-02-18T11:48:45.581 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:45.588+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f20f80b5dc0 con 0x7f20f80b9370 2024-02-18T11:48:46.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.282+0000 7f20dffff640 1 -- 172.21.15.116:0/2584440450 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7f20e8058910 con 0x7f20f80b9370 2024-02-18T11:48:46.276 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2024-02-18T11:48:46.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.284+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f20e406bc70 msgr2=0x7f20e406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:46.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.284+0000 7f20fd823640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f20e406bc70 0x7f20e406e130 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f20ec036000 tx=0x7f20ec03c040 comp rx=0 tx=0).stop 2024-02-18T11:48:46.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.284+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 msgr2=0x7f20f80b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:46.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.284+0000 7f20fd823640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80b7b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c7f0 tx=0x7f20e800ccb0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 shutdown_connections 2024-02-18T11:48:46.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f20e406bc70 0x7f20e406e130 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20f815dbf0 0x7f20f8151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20f80cec80 0x7f20f815f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 --2- 172.21.15.116:0/2584440450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f80b9370 0x7f20f80b7b50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 >> 172.21.15.116:0/2584440450 conn(0x7f20f80b5480 msgr2=0x7f20f80b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:46.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 shutdown_connections 2024-02-18T11:48:46.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.285+0000 7f20fd823640 1 -- 172.21.15.116:0/2584440450 wait complete. 2024-02-18T11:48:46.298 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-18T11:48:46.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.410+0000 7fce2f9bc640 1 Processor -- start 2024-02-18T11:48:46.403 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.410+0000 7fce2f9bc640 1 -- start start 2024-02-18T11:48:46.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2f9bc640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fce280b31a0 0x7fce2806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:46.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2f9bc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce2806e870 0x7fce2815e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:46.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2f9bc640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce280b3610 con 0x7fce28144090 2024-02-18T11:48:46.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2f9bc640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2806ece0 con 0x7fce280b31a0 2024-02-18T11:48:46.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2f9bc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce280b2a50 con 0x7fce2806e870 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2cf57640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fce280b31a0 0x7fce2806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2df59640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce2806e870 0x7fce2815e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.411+0000 7fce2d758640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fce28144090 0x7fce280cc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:42924/0 (socket says 172.21.15.116:42924) 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce2d758640 1 -- 172.21.15.116:0/390648200 learned_addr learned my addr 172.21.15.116:0/390648200 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce27fff640 1 -- 172.21.15.116:0/390648200 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869939638 0 0) 0x7fce280b3610 con 0x7fce28144090 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce27fff640 1 -- 172.21.15.116:0/390648200 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce10003520 con 0x7fce28144090 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce2cf57640 1 -- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce2806e870 msgr2=0x7fce2815e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce2cf57640 1 --2- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce2806e870 0x7fce2815e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce2cf57640 1 -- 172.21.15.116:0/390648200 >> v1:172.21.15.143:6789/0 conn(0x7fce28144090 legacy=0x7fce280cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce2cf57640 1 -- 172.21.15.116:0/390648200 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce280cd0f0 con 0x7fce280b31a0 2024-02-18T11:48:46.407 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.412+0000 7fce2df59640 1 --2- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce2806e870 0x7fce2815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2cf57640 1 --2- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce280b31a0 0x7fce2806e330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0097d0 tx=0x7fce1c02b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f6c5ecaa9e3930 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce27fff640 1 -- 172.21.15.116:0/390648200 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c031770 con 0x7fce280b31a0 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce27fff640 1 -- 172.21.15.116:0/390648200 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c0318d0 con 0x7fce280b31a0 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce27fff640 1 -- 172.21.15.116:0/390648200 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c031c60 con 0x7fce280b31a0 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2f9bc640 1 -- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce280b31a0 msgr2=0x7fce2806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce280b31a0 0x7fce2806e330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0097d0 tx=0x7fce1c02b810 comp rx=0 tx=0).stop 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2f9bc640 1 -- 172.21.15.116:0/390648200 shutdown_connections 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce2806e870 0x7fce2815e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/390648200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce280b31a0 0x7fce2806e330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.413+0000 7fce2f9bc640 1 -- 172.21.15.116:0/390648200 >> 172.21.15.116:0/390648200 conn(0x7fce28067000 msgr2=0x7fce2806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.414+0000 7fce2f9bc640 1 -- 172.21.15.116:0/390648200 shutdown_connections 2024-02-18T11:48:46.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.414+0000 7fce2f9bc640 1 -- 172.21.15.116:0/390648200 wait complete. 2024-02-18T11:48:46.410 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 Processor -- start 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 -- start start 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce2806e870 0x7fce2815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 0x7fce28142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce28144090 0x7fce28166da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28163a40 con 0x7fce280b31a0 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28143110 con 0x7fce2806e870 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.415+0000 7fce2f9bc640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce281433f0 con 0x7fce28144090 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 0x7fce28142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 0x7fce28142700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:36950/0 (socket says 172.21.15.116:36950) 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 -- 172.21.15.116:0/4265684630 learned_addr learned my addr 172.21.15.116:0/4265684630 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2d758640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce2806e870 0x7fce2815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:46.411 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2df59640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce28144090 0x7fce28166da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 -- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce28144090 msgr2=0x7fce28166da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce28144090 0x7fce28166da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 -- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce2806e870 msgr2=0x7fce2815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce2806e870 0x7fce2815a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 -- 172.21.15.116:0/4265684630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20009a70 con 0x7fce280b31a0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2cf57640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 0x7fce28142700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0097d0 tx=0x7fce1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c032610 con 0x7fce280b31a0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c032790 con 0x7fce280b31a0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1c009400 con 0x7fce280b31a0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.416+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c032b40 con 0x7fce280b31a0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.417+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce28142c40 con 0x7fce280b31a0 2024-02-18T11:48:46.412 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.418+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce280cbe40 con 0x7fce280b31a0 2024-02-18T11:48:46.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.418+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fce1c032ca0 con 0x7fce280b31a0 2024-02-18T11:48:46.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.418+0000 7fce25ffb640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fce1406bc70 0x7fce1406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:46.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.419+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fce1c0ac640 con 0x7fce280b31a0 2024-02-18T11:48:46.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.419+0000 7fce2d758640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fce1406bc70 0x7fce1406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:46.413 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.419+0000 7fce2d758640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fce1406bc70 0x7fce1406e130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce18005880 tx=0x7fce18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:46.416 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.423+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce1c076cf0 con 0x7fce280b31a0 2024-02-18T11:48:46.626 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:46.632+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fce28143910 con 0x7fce280b31a0 2024-02-18T11:48:47.290 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.297+0000 7fce25ffb640 1 -- 172.21.15.116:0/4265684630 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fce1c0aae60 con 0x7fce280b31a0 2024-02-18T11:48:47.290 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 5 and data pool 6 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.300+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fce1406bc70 msgr2=0x7fce1406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.300+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fce1406bc70 0x7fce1406e130 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fce18005880 tx=0x7fce18009040 comp rx=0 tx=0).stop 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.300+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 msgr2=0x7fce28142700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.300+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 0x7fce28142700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0097d0 tx=0x7fce1c0046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 shutdown_connections 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fce1406bc70 0x7fce1406e130 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fce28144090 0x7fce28166da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.297 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce280b31a0 0x7fce28142700 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 --2- 172.21.15.116:0/4265684630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fce2806e870 0x7fce2815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 >> 172.21.15.116:0/4265684630 conn(0x7fce28067000 msgr2=0x7fce2806e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:47.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.301+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 shutdown_connections 2024-02-18T11:48:47.298 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.302+0000 7fce2f9bc640 1 -- 172.21.15.116:0/4265684630 wait complete. 2024-02-18T11:48:47.322 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:47.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.429+0000 7f0318476640 1 Processor -- start 2024-02-18T11:48:47.422 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.429+0000 7f0318476640 1 -- start start 2024-02-18T11:48:47.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0318476640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f031015ac40 0x7f031006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0318476640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 0x7f03100b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0318476640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03100b02e0 con 0x7f031015a860 2024-02-18T11:48:47.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0318476640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031006ece0 con 0x7f031015ac40 2024-02-18T11:48:47.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0318476640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03100af6d0 con 0x7f031006e850 2024-02-18T11:48:47.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0316212640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f031015a860 0x7f03100cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34718/0 (socket says 172.21.15.116:34718) 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0316212640 1 -- 172.21.15.116:0/1959111462 learned_addr learned my addr 172.21.15.116:0/1959111462 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315a11640 1 --2- 172.21.15.116:0/1959111462 >> v2:172.21.15.143:3300/0 conn(0x7f031015ac40 0x7f031006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0316a13640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 0x7f03100b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119115348 0 0) 0x7f03100b02e0 con 0x7f031015a860 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02fc003520 con 0x7f031015a860 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101493781 0 0) 0x7f02fc003520 con 0x7f031015a860 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 msgr2=0x7f03100b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 0x7f03100b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 >> v2:172.21.15.143:3300/0 conn(0x7f031015ac40 msgr2=0x7f031006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 --2- 172.21.15.116:0/1959111462 >> v2:172.21.15.143:3300/0 conn(0x7f031015ac40 0x7f031006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03100af9b0 con 0x7f031015a860 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f030c001670 con 0x7f031015a860 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 >> v1:172.21.15.116:6789/0 conn(0x7f031015a860 legacy=0x7f03100cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 0x7f02fc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02fc003ef0 0x7f02fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02fc004d00 0x7f02fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031006ece0 con 0x7f02fc003ef0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100af6d0 con 0x7f02fc004d00 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.430+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc00b980 con 0x7f031006e850 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0315a11640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02fc003ef0 0x7f02fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0316a13640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02fc004d00 0x7f02fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0316212640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 0x7f02fc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0315a11640 1 -- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 msgr2=0x7f02fc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0315a11640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f031006e850 0x7f02fc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0315a11640 1 -- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02fc004d00 msgr2=0x7f02fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0315a11640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02fc004d00 0x7f02fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.431+0000 7f0315a11640 1 -- 172.21.15.116:0/1959111462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc003520 con 0x7f02fc003ef0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0315a11640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02fc003ef0 0x7f02fc0047c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0300009a90 tx=0x7f02fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7358c45344213933 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030002c6b0 con 0x7f02fc003ef0 2024-02-18T11:48:47.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030002c810 con 0x7f02fc003ef0 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0315210640 1 -- 172.21.15.116:0/1959111462 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030002cba0 con 0x7f02fc003ef0 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0318476640 1 -- 172.21.15.116:0/1959111462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02fc003ef0 msgr2=0x7f02fc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0318476640 1 --2- 172.21.15.116:0/1959111462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02fc003ef0 0x7f02fc0047c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0300009a90 tx=0x7f02fc0057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0316a13640 1 -- 172.21.15.116:0/1959111462 reap_dead start 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0318476640 1 -- 172.21.15.116:0/1959111462 shutdown_connections 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0318476640 1 -- 172.21.15.116:0/1959111462 >> 172.21.15.116:0/1959111462 conn(0x7f0310066f20 msgr2=0x7f031006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0318476640 1 -- 172.21.15.116:0/1959111462 shutdown_connections 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.432+0000 7f0318476640 1 -- 172.21.15.116:0/1959111462 wait complete. 2024-02-18T11:48:47.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.433+0000 7f0318476640 1 Processor -- start 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.433+0000 7f0318476640 1 -- start start 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0318476640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02fc003ef0 0x7f0310152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0318476640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0310156450 0x7f03101414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0318476640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 0x7f031006d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0318476640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03101568d0 con 0x7f0310151530 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0318476640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0310141a00 con 0x7f0310156450 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0318476640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0310151d60 con 0x7f02fc003ef0 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 0x7f031006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 0x7f031006d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41838/0 (socket says 172.21.15.116:41838) 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 -- 172.21.15.116:0/319618696 learned_addr learned my addr 172.21.15.116:0/319618696 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:47.433 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0315a11640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0310156450 0x7f03101414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316212640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02fc003ef0 0x7f0310152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 -- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02fc003ef0 msgr2=0x7f0310152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02fc003ef0 0x7f0310152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 -- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0310156450 msgr2=0x7f03101414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0310156450 0x7f03101414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.434+0000 7f0316a13640 1 -- 172.21.15.116:0/319618696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0310152040 con 0x7f0310151530 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.435+0000 7f0316a13640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 0x7f031006d200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f031015b020 tx=0x7f030800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:47.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.435+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800e040 con 0x7f0310151530 2024-02-18T11:48:47.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.435+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030800f040 con 0x7f0310151530 2024-02-18T11:48:47.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.435+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0310152320 con 0x7f0310151530 2024-02-18T11:48:47.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.435+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308015430 con 0x7f0310151530 2024-02-18T11:48:47.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.435+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03101528e0 con 0x7f0310151530 2024-02-18T11:48:47.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.436+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0310152b00 con 0x7f0310151530 2024-02-18T11:48:47.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.436+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f0308015590 con 0x7f0310151530 2024-02-18T11:48:47.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.437+0000 7f0306ffd640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f02f406bc70 0x7f02f406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.437+0000 7f0316212640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f02f406bc70 0x7f02f406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.437+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f030808b210 con 0x7f0310151530 2024-02-18T11:48:47.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.437+0000 7f0316212640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f02f406bc70 0x7f02f406e130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f030c001050 tx=0x7f030c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:47.437 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.441+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03080558f0 con 0x7f0310151530 2024-02-18T11:48:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.611+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0310074520 con 0x7f0310151530 2024-02-18T11:48:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.612+0000 7f0306ffd640 1 -- 172.21.15.116:0/319618696 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16967 (secure 0 0 0) 0x7f0308058fb0 con 0x7f0310151530 2024-02-18T11:48:47.606 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:47.606 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:46.634604+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:48:44.953507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-02-18T11:48:45.607007+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f02f406bc70 msgr2=0x7f02f406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f02f406bc70 0x7f02f406e130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f030c001050 tx=0x7f030c009040 comp rx=0 tx=0).stop 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 msgr2=0x7f031006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 0x7f031006d200 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f031015b020 tx=0x7f030800f580 comp rx=0 tx=0).stop 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 shutdown_connections 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f02f406bc70 0x7f02f406e130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0310151530 0x7f031006d200 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0310156450 0x7f03101414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 --2- 172.21.15.116:0/319618696 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02fc003ef0 0x7f0310152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 >> 172.21.15.116:0/319618696 conn(0x7f0310066f20 msgr2=0x7f0310156fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 shutdown_connections 2024-02-18T11:48:47.609 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.614+0000 7f0318476640 1 -- 172.21.15.116:0/319618696 wait complete. 2024-02-18T11:48:47.619 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.726+0000 7f0b2b0ae640 1 Processor -- start 2024-02-18T11:48:47.719 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.726+0000 7f0b2b0ae640 1 -- start start 2024-02-18T11:48:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b2b0ae640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b2415a860 0x7f0b240cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b2b0ae640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0b240cd1f0 0x7f0b2406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b2b0ae640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b240b02e0 con 0x7f0b2415ac40 2024-02-18T11:48:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b2b0ae640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240b1780 con 0x7f0b240cd1f0 2024-02-18T11:48:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b2b0ae640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240af6d0 con 0x7f0b2415a860 2024-02-18T11:48:47.721 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0b240cd1f0 0x7f0b2406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0b240cd1f0 0x7f0b2406e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41850/0 (socket says 172.21.15.116:41850) 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 -- 172.21.15.116:0/2275456379 learned_addr learned my addr 172.21.15.116:0/2275456379 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b28e4a640 1 --2- 172.21.15.116:0/2275456379 >> v2:172.21.15.143:3300/0 conn(0x7f0b2415a860 0x7f0b240cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 -- 172.21.15.116:0/2275456379 >> v2:172.21.15.143:3300/0 conn(0x7f0b2415a860 msgr2=0x7f0b240cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 --2- 172.21.15.116:0/2275456379 >> v2:172.21.15.143:3300/0 conn(0x7f0b2415a860 0x7f0b240cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 -- 172.21.15.116:0/2275456379 >> v1:172.21.15.143:6789/0 conn(0x7f0b2415ac40 legacy=0x7f0b2415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b23fff640 1 -- 172.21.15.116:0/2275456379 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240af9b0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.727+0000 7f0b28e4a640 1 --2- 172.21.15.116:0/2275456379 >> v2:172.21.15.143:3300/0 conn(0x7f0b2415a860 0x7f0b240cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.728+0000 7f0b23fff640 1 --2- 172.21.15.116:0/2275456379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2406e310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b2406ef20 tx=0x7f0b1002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedd564eef293766 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.728+0000 7f0b237fe640 1 -- 172.21.15.116:0/2275456379 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10036440 con 0x7f0b240cd1f0 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.728+0000 7f0b237fe640 1 -- 172.21.15.116:0/2275456379 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100365a0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.728+0000 7f0b237fe640 1 -- 172.21.15.116:0/2275456379 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10036930 con 0x7f0b240cd1f0 2024-02-18T11:48:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/2275456379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 msgr2=0x7f0b2406e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/2275456379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2406e310 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b2406ef20 tx=0x7f0b1002e170 comp rx=0 tx=0).stop 2024-02-18T11:48:47.725 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/2275456379 shutdown_connections 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/2275456379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2406e310 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/2275456379 >> v2:172.21.15.143:3300/0 conn(0x7f0b2415a860 0x7f0b240cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/2275456379 >> 172.21.15.116:0/2275456379 conn(0x7f0b24066f20 msgr2=0x7f0b2406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/2275456379 shutdown_connections 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.729+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/2275456379 wait complete. 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.730+0000 7f0b2b0ae640 1 Processor -- start 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.730+0000 7f0b2b0ae640 1 -- start start 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2414af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b2415a860 0x7f0b24157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0b2415ac40 0x7f0b241582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2414b4c0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240ad5f0 con 0x7f0b2415a860 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240ad8d0 con 0x7f0b2415ac40 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2414af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.726 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2414af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41860/0 (socket says 172.21.15.116:41860) 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 -- 172.21.15.116:0/1898006167 learned_addr learned my addr 172.21.15.116:0/1898006167 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b23fff640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b2415a860 0x7f0b24157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2964b640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0b2415ac40 0x7f0b241582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 -- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0b2415ac40 msgr2=0x7f0b241582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0b2415ac40 0x7f0b241582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 -- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b2415a860 msgr2=0x7f0b24157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b2415a860 0x7f0b24157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 -- 172.21.15.116:0/1898006167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1000b770 con 0x7f0b240cd1f0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b23fff640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b2415a860 0x7f0b24157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b28e4a640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2414af80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800e7e0 tx=0x7f0b1800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004030 con 0x7f0b240cd1f0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18004190 con 0x7f0b240cd1f0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b240adbb0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18004520 con 0x7f0b240cd1f0 2024-02-18T11:48:47.727 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.731+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240ae170 con 0x7f0b240cd1f0 2024-02-18T11:48:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.732+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b24074520 con 0x7f0b240cd1f0 2024-02-18T11:48:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.733+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f0b1800c6d0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.733+0000 7f0b217fa640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0b0806bc70 0x7f0b0806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:47.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.733+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b1808a6f0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.733+0000 7f0b23fff640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0b0806bc70 0x7f0b0806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:47.729 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.734+0000 7f0b23fff640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0b0806bc70 0x7f0b0806e130 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b10038000 tx=0x7f0b10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:47.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.737+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b18054cc0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.903 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.910+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b240cf2c0 con 0x7f0b240cd1f0 2024-02-18T11:48:47.904 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.911+0000 7f0b217fa640 1 -- 172.21.15.116:0/1898006167 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16967 (secure 0 0 0) 0x7f0b18058380 con 0x7f0b240cd1f0 2024-02-18T11:48:47.904 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:47.905 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:46.634604+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:48:44.953507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-02-18T11:48:45.607007+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:47.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0b0806bc70 msgr2=0x7f0b0806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0b0806bc70 0x7f0b0806e130 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0b10038000 tx=0x7f0b10008040 comp rx=0 tx=0).stop 2024-02-18T11:48:47.906 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 msgr2=0x7f0b2414af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2414af80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800e7e0 tx=0x7f0b1800eca0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 shutdown_connections 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f0b0806bc70 0x7f0b0806e130 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0b2415ac40 0x7f0b241582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b2415a860 0x7f0b24157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 --2- 172.21.15.116:0/1898006167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b240cd1f0 0x7f0b2414af80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 >> 172.21.15.116:0/1898006167 conn(0x7f0b24066f20 msgr2=0x7f0b240b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 shutdown_connections 2024-02-18T11:48:47.907 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:47.913+0000 7f0b2b0ae640 1 -- 172.21.15.116:0/1898006167 wait complete. 2024-02-18T11:48:47.917 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-18T11:48:48.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.026+0000 7f245cc2a640 1 Processor -- start 2024-02-18T11:48:48.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.026+0000 7f245cc2a640 1 -- start start 2024-02-18T11:48:48.021 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f245cc2a640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f245815dbf0 0x7f24581617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f245cc2a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f24580ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f245cc2a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24580cec80 0x7f245806c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f245cc2a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24580cf110 con 0x7f24580cec80 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f245cc2a640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24580d1ee0 con 0x7f245815dbf0 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f245cc2a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2458161dc0 con 0x7f24580b9370 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f2456d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24580cec80 0x7f245806c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f2456d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24580cec80 0x7f245806c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41864/0 (socket says 172.21.15.116:41864) 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.027+0000 7f2456d9d640 1 -- 172.21.15.116:0/1160408018 learned_addr learned my addr 172.21.15.116:0/1160408018 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f245659c640 1 --2- 172.21.15.116:0/1160408018 >> v2:172.21.15.143:3300/0 conn(0x7f245815dbf0 0x7f24581617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2455d9b640 1 --2- 172.21.15.116:0/1160408018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f24580ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2456d9d640 1 -- 172.21.15.116:0/1160408018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 msgr2=0x7f24580ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2456d9d640 1 --2- 172.21.15.116:0/1160408018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f24580ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2456d9d640 1 -- 172.21.15.116:0/1160408018 >> v2:172.21.15.143:3300/0 conn(0x7f245815dbf0 msgr2=0x7f24581617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:48.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2456d9d640 1 --2- 172.21.15.116:0/1160408018 >> v2:172.21.15.143:3300/0 conn(0x7f245815dbf0 0x7f24581617e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2456d9d640 1 -- 172.21.15.116:0/1160408018 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24581620a0 con 0x7f24580cec80 2024-02-18T11:48:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f245659c640 1 --2- 172.21.15.116:0/1160408018 >> v2:172.21.15.143:3300/0 conn(0x7f245815dbf0 0x7f24581617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.028+0000 7f2456d9d640 1 --2- 172.21.15.116:0/1160408018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f245806c210 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f245806d330 tx=0x7f245002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73f30fb0e5806d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245559a640 1 -- 172.21.15.116:0/1160408018 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450031870 con 0x7f24580cec80 2024-02-18T11:48:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245559a640 1 -- 172.21.15.116:0/1160408018 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24500319d0 con 0x7f24580cec80 2024-02-18T11:48:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245559a640 1 -- 172.21.15.116:0/1160408018 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450031d60 con 0x7f24580cec80 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160408018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 msgr2=0x7f245806c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160408018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f245806c210 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f245806d330 tx=0x7f245002b910 comp rx=0 tx=0).stop 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160408018 shutdown_connections 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160408018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f24580ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160408018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f245806c210 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160408018 >> v2:172.21.15.143:3300/0 conn(0x7f245815dbf0 0x7f24581617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160408018 >> 172.21.15.116:0/1160408018 conn(0x7f24580b5480 msgr2=0x7f24580b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160408018 shutdown_connections 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.029+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160408018 wait complete. 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.030+0000 7f245cc2a640 1 Processor -- start 2024-02-18T11:48:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 -- start start 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f2458160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f2458151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f245815dbf0 0x7f24580cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245815f600 con 0x7f24580cec80 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2458153540 con 0x7f245815dbf0 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245cc2a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24580ccde0 con 0x7f24580b9370 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f2455d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f2458151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f2455d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f2458151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41880/0 (socket says 172.21.15.116:41880) 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f2455d9b640 1 -- 172.21.15.116:0/1160850874 learned_addr learned my addr 172.21.15.116:0/1160850874 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f245659c640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f2458160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.031+0000 7f2456d9d640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f245815dbf0 0x7f24580cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2455d9b640 1 -- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 msgr2=0x7f2458160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2455d9b640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f2458160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2455d9b640 1 -- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f245815dbf0 msgr2=0x7f24580cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2455d9b640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f245815dbf0 0x7f24580cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2455d9b640 1 -- 172.21.15.116:0/1160850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500094d0 con 0x7f24580cec80 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2455d9b640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f2458151c10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d800 tx=0x7f244c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0096a0 con 0x7f24580cec80 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c009800 con 0x7f24580cec80 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245815e150 con 0x7f24580cec80 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c009b90 con 0x7f24580cec80 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.032+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24580cd3a0 con 0x7f24580cec80 2024-02-18T11:48:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.033+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2458067840 con 0x7f24580cec80 2024-02-18T11:48:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.033+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f244c009cf0 con 0x7f24580cec80 2024-02-18T11:48:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.034+0000 7f2446ffd640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f243806bc70 0x7f243806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.034+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f244c08af20 con 0x7f24580cec80 2024-02-18T11:48:48.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.034+0000 7f245659c640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f243806bc70 0x7f243806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.034+0000 7f245659c640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f243806bc70 0x7f243806e130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2448009fd0 tx=0x7f24480097f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:48.031 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.038+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f244c055620 con 0x7f24580cec80 2024-02-18T11:48:48.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.244+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2458074520 con 0x7f24580cec80 2024-02-18T11:48:48.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.312+0000 7f2446ffd640 1 -- 172.21.15.116:0/1160850874 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f244c00b440 con 0x7f24580cec80 2024-02-18T11:48:48.307 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f243806bc70 msgr2=0x7f243806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f243806bc70 0x7f243806e130 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2448009fd0 tx=0x7f24480097f0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 msgr2=0x7f2458151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f2458151c10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f244c00d800 tx=0x7f244c00dcc0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 shutdown_connections 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f243806bc70 0x7f243806e130 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f245815dbf0 0x7f24580cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24580cec80 0x7f2458151c10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 --2- 172.21.15.116:0/1160850874 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f24580b9370 0x7f2458160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 >> 172.21.15.116:0/1160850874 conn(0x7f24580b5480 msgr2=0x7f245815ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 shutdown_connections 2024-02-18T11:48:48.308 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.314+0000 7f245cc2a640 1 -- 172.21.15.116:0/1160850874 wait complete. 2024-02-18T11:48:48.318 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-02-18T11:48:48.319 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-02-18T11:48:48.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 Processor -- start 2024-02-18T11:48:48.423 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 -- start start 2024-02-18T11:48:48.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c900b1d80 0x7f4c900ae920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900ae490 con 0x7f4c9015b7b0 2024-02-18T11:48:48.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900b2570 con 0x7f4c900b1d80 2024-02-18T11:48:48.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.430+0000 7f4c955a0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b2850 con 0x7f4c9015fb20 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c94d9f640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c900b1d80 0x7f4c900ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c8f7fe640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c8ffff640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4c9015b7b0 0x7f4c9015f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.116:50886/0 (socket says 172.21.15.116:50886) 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c8ffff640 1 -- 172.21.15.116:0/205804994 learned_addr learned my addr 172.21.15.116:0/205804994 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c94d9f640 1 -- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 msgr2=0x7f4c900b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c94d9f640 1 --2- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c94d9f640 1 -- 172.21.15.116:0/205804994 >> v1:172.21.15.143:6789/0 conn(0x7f4c9015b7b0 legacy=0x7f4c9015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.431+0000 7f4c94d9f640 1 -- 172.21.15.116:0/205804994 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900b2b30 con 0x7f4c900b1d80 2024-02-18T11:48:48.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.432+0000 7f4c94d9f640 1 --2- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c900b1d80 0x7f4c900ae920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009d90 tx=0x7f4c8802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6879d8248cd4db76 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:48.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.432+0000 7f4c8effd640 1 -- 172.21.15.116:0/205804994 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8802cc50 con 0x7f4c900b1d80 2024-02-18T11:48:48.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.432+0000 7f4c8effd640 1 -- 172.21.15.116:0/205804994 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8802cdb0 con 0x7f4c900b1d80 2024-02-18T11:48:48.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.432+0000 7f4c8effd640 1 -- 172.21.15.116:0/205804994 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88034660 con 0x7f4c900b1d80 2024-02-18T11:48:48.426 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 -- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c900b1d80 msgr2=0x7f4c900ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 --2- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c900b1d80 0x7f4c900ae920 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4c88009d90 tx=0x7f4c8802bea0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 -- 172.21.15.116:0/205804994 shutdown_connections 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 --2- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c900b1d80 0x7f4c900ae920 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 --2- 172.21.15.116:0/205804994 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 -- 172.21.15.116:0/205804994 >> 172.21.15.116:0/205804994 conn(0x7f4c900d1630 msgr2=0x7f4c900d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 -- 172.21.15.116:0/205804994 shutdown_connections 2024-02-18T11:48:48.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.433+0000 7f4c955a0640 1 -- 172.21.15.116:0/205804994 wait complete. 2024-02-18T11:48:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.434+0000 7f4c955a0640 1 Processor -- start 2024-02-18T11:48:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.434+0000 7f4c955a0640 1 -- start start 2024-02-18T11:48:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c955a0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 0x7f4c9014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c955a0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9015b7b0 0x7f4c9015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c955a0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c955a0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9014fd30 con 0x7f4c900b1d80 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c955a0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9015c2d0 con 0x7f4c9015b7b0 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c955a0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9015c5b0 con 0x7f4c9015fb20 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 0x7f4c9014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 0x7f4c9014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41884/0 (socket says 172.21.15.116:41884) 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 -- 172.21.15.116:0/2148061284 learned_addr learned my addr 172.21.15.116:0/2148061284 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8f7fe640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9015b7b0 0x7f4c9015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c94d9f640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 -- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 msgr2=0x7f4c900cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 -- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9015b7b0 msgr2=0x7f4c9015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9015b7b0 0x7f4c9015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 -- 172.21.15.116:0/2148061284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88009a70 con 0x7f4c900b1d80 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8f7fe640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9015b7b0 0x7f4c9015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:48.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c94d9f640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:48.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8ffff640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 0x7f4c9014f7f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c400 tx=0x7f4c8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:48.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.435+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80005ce0 con 0x7f4c900b1d80 2024-02-18T11:48:48.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.436+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80005e40 con 0x7f4c900b1d80 2024-02-18T11:48:48.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.436+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9015c890 con 0x7f4c900b1d80 2024-02-18T11:48:48.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.436+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800072d0 con 0x7f4c900b1d80 2024-02-18T11:48:48.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.436+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9015ce50 con 0x7f4c900b1d80 2024-02-18T11:48:48.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.436+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9015d040 con 0x7f4c900b1d80 2024-02-18T11:48:48.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.437+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f4c80007430 con 0x7f4c900b1d80 2024-02-18T11:48:48.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.437+0000 7f4c8cff9640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4c7006bca0 0x7f4c7006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:48.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.437+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c80089fd0 con 0x7f4c900b1d80 2024-02-18T11:48:48.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.437+0000 7f4c8f7fe640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4c7006bca0 0x7f4c7006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:48.432 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.438+0000 7f4c8f7fe640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4c7006bca0 0x7f4c7006e160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c90071270 tx=0x7f4c78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:48.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.442+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c800545a0 con 0x7f4c900b1d80 2024-02-18T11:48:48.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:48.648+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4c90074520 con 0x7f4c900b1d80 2024-02-18T11:48:49.312 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.319+0000 7f4c8cff9640 1 -- 172.21.15.116:0/2148061284 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c80057c60 con 0x7f4c900b1d80 2024-02-18T11:48:49.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4c7006bca0 msgr2=0x7f4c7006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4c7006bca0 0x7f4c7006e160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c90071270 tx=0x7f4c78008040 comp rx=0 tx=0).stop 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 msgr2=0x7f4c9014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 0x7f4c9014f7f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c400 tx=0x7f4c8000c8c0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 shutdown_connections 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f4c7006bca0 0x7f4c7006e160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c9015fb20 0x7f4c900cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c9015b7b0 0x7f4c9015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 --2- 172.21.15.116:0/2148061284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c900b1d80 0x7f4c9014f7f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 >> 172.21.15.116:0/2148061284 conn(0x7f4c900d1630 msgr2=0x7f4c90159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 shutdown_connections 2024-02-18T11:48:49.315 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.321+0000 7f4c955a0640 1 -- 172.21.15.116:0/2148061284 wait complete. 2024-02-18T11:48:49.326 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-02-18T11:48:49.326 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:49.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.433+0000 7f2b5a0df640 1 Processor -- start 2024-02-18T11:48:49.427 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.433+0000 7f2b5a0df640 1 -- start start 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b5a0df640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b5415ae40 0x7f2b540b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b5a0df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b540cefe0 0x7f2b5406d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b5a0df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b540cdff0 con 0x7f2b540b0210 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b5a0df640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b540b1070 con 0x7f2b5415ae40 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b5a0df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540b1ea0 con 0x7f2b540cefe0 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b52ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b5415ae40 0x7f2b540b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b52ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b5415ae40 0x7f2b540b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41888/0 (socket says 172.21.15.116:41888) 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.434+0000 7f2b52ffd640 1 -- 172.21.15.116:0/955575561 learned_addr learned my addr 172.21.15.116:0/955575561 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.435+0000 7f2b52ffd640 1 -- 172.21.15.116:0/955575561 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b540cefe0 msgr2=0x7f2b5406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.435+0000 7f2b52ffd640 1 --2- 172.21.15.116:0/955575561 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b540cefe0 0x7f2b5406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.435+0000 7f2b52ffd640 1 -- 172.21.15.116:0/955575561 >> v1:172.21.15.143:6789/0 conn(0x7f2b540b0210 legacy=0x7f2b540b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:49.429 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.435+0000 7f2b52ffd640 1 -- 172.21.15.116:0/955575561 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5415b3e0 con 0x7f2b5415ae40 2024-02-18T11:48:49.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.435+0000 7f2b52ffd640 1 --2- 172.21.15.116:0/955575561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b5415ae40 0x7f2b540b1960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009870 tx=0x7f2b4802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5986715faf604e server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:49.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b527fc640 1 -- 172.21.15.116:0/955575561 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48031640 con 0x7f2b5415ae40 2024-02-18T11:48:49.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b527fc640 1 -- 172.21.15.116:0/955575561 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b480317a0 con 0x7f2b5415ae40 2024-02-18T11:48:49.430 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b527fc640 1 -- 172.21.15.116:0/955575561 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48031b30 con 0x7f2b5415ae40 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 -- 172.21.15.116:0/955575561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b5415ae40 msgr2=0x7f2b540b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/955575561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b5415ae40 0x7f2b540b1960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009870 tx=0x7f2b4802b8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 -- 172.21.15.116:0/955575561 shutdown_connections 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/955575561 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b540cefe0 0x7f2b5406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/955575561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b5415ae40 0x7f2b540b1960 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 -- 172.21.15.116:0/955575561 >> 172.21.15.116:0/955575561 conn(0x7f2b54066f20 msgr2=0x7f2b5406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 -- 172.21.15.116:0/955575561 shutdown_connections 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.436+0000 7f2b5a0df640 1 -- 172.21.15.116:0/955575561 wait complete. 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.437+0000 7f2b5a0df640 1 Processor -- start 2024-02-18T11:48:49.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 -- start start 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 0x7f2b54141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b540cefe0 0x7f2b54157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b5415ae40 0x7f2b54157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54141670 con 0x7f2b540b0210 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b54154ab0 con 0x7f2b540cefe0 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b5a0df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b540ac390 con 0x7f2b5415ae40 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 0x7f2b54141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b537fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 0x7f2b54141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41894/0 (socket says 172.21.15.116:41894) 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.438+0000 7f2b537fe640 1 -- 172.21.15.116:0/2924587918 learned_addr learned my addr 172.21.15.116:0/2924587918 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b52ffd640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b540cefe0 0x7f2b54157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b537fe640 1 -- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b5415ae40 msgr2=0x7f2b54157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b537fe640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b5415ae40 0x7f2b54157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b537fe640 1 -- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b540cefe0 msgr2=0x7f2b54157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b537fe640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b540cefe0 0x7f2b54157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b537fe640 1 -- 172.21.15.116:0/2924587918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480094d0 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b52ffd640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b540cefe0 0x7f2b54157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b537fe640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 0x7f2b54141130 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400b3e0 tx=0x7f2b4400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44011830 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b440119b0 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b540a93e0 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44011d60 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.439+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b54154d00 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.440+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b54074520 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.440+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f2b4401a400 con 0x7f2b540b0210 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.441+0000 7f2b3bfff640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2b4006bca0 0x7f2b4006e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.435 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.441+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b4408a9e0 con 0x7f2b540b0210 2024-02-18T11:48:49.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.441+0000 7f2b52ffd640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2b4006bca0 0x7f2b4006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.436 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.441+0000 7f2b52ffd640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2b4006bca0 0x7f2b4006e160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b48036000 tx=0x7f2b4803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:49.438 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.445+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b440550c0 con 0x7f2b540b0210 2024-02-18T11:48:49.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.650+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b5415cd60 con 0x7f2b540b0210 2024-02-18T11:48:49.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.651+0000 7f2b3bfff640 1 -- 172.21.15.116:0/2924587918 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f2b44015090 con 0x7f2b540b0210 2024-02-18T11:48:49.644 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:49.645 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/673479819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":673479819},{"type":"v1","addr":"172.21.15.143:6835","nonce":673479819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3865796101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3865796101},{"type":"v1","addr":"172.21.15.143:6834","nonce":3865796101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/4242183425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":4242183425},{"type":"v1","addr":"172.21.15.116:6837","nonce":4242183425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:48:46.634911+0000","modified":"2024-02-18T11:48:48.649669+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1597642942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1597642942},{"type":"v1","addr":"172.21.15.116:6836","nonce":1597642942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-18T11:48:49.645 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 15 2024-02-18T11:48:49.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2b4006bca0 msgr2=0x7f2b4006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2b4006bca0 0x7f2b4006e160 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b48036000 tx=0x7f2b4803c040 comp rx=0 tx=0).stop 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 msgr2=0x7f2b54141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 0x7f2b54141130 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400b3e0 tx=0x7f2b4400b8a0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 shutdown_connections 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f2b4006bca0 0x7f2b4006e160 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b5415ae40 0x7f2b54157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b540cefe0 0x7f2b54157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 --2- 172.21.15.116:0/2924587918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b540b0210 0x7f2b54141130 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 >> 172.21.15.116:0/2924587918 conn(0x7f2b54066f20 msgr2=0x7f2b54158010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 shutdown_connections 2024-02-18T11:48:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.653+0000 7f2b5a0df640 1 -- 172.21.15.116:0/2924587918 wait complete. 2024-02-18T11:48:49.657 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T11:48:49.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.764+0000 7f06cc102640 1 Processor -- start 2024-02-18T11:48:49.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.765+0000 7f06cc102640 1 -- start start 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.765+0000 7f06cc102640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f06c40d0730 0x7f06c40cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.765+0000 7f06cc102640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c40ccb70 con 0x7f06c40d0b90 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06cc102640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c40cf970 con 0x7f06c40b0750 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06cc102640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c40cfc50 con 0x7f06c40d0730 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06c969d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f06c40d0730 0x7f06c40cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06ca69f640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f06c40d0b90 0x7f06c40cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34730/0 (socket says 172.21.15.116:34730) 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06ca69f640 1 -- 172.21.15.116:0/869622577 learned_addr learned my addr 172.21.15.116:0/869622577 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:49.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06c969d640 1 -- 172.21.15.116:0/869622577 >> v1:172.21.15.143:6789/0 conn(0x7f06c40b0750 legacy=0x7f06c40b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:49.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06c969d640 1 -- 172.21.15.116:0/869622577 >> v1:172.21.15.116:6789/0 conn(0x7f06c40d0b90 legacy=0x7f06c40cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06c969d640 1 -- 172.21.15.116:0/869622577 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40cff30 con 0x7f06c40d0730 2024-02-18T11:48:49.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.766+0000 7f06c8e9c640 1 -- 172.21.15.116:0/869622577 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195585575 0 0) 0x7f06c40ccb70 con 0x7f06c40d0b90 2024-02-18T11:48:49.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06c969d640 1 --2- 172.21.15.116:0/869622577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c40cbe40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f06b40097a0 tx=0x7f06b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5031d8c2b135eb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:49.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06c8e9c640 1 -- 172.21.15.116:0/869622577 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4031830 con 0x7f06c40d0730 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06c8e9c640 1 -- 172.21.15.116:0/869622577 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b4031990 con 0x7f06c40d0730 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06c8e9c640 1 -- 172.21.15.116:0/869622577 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4031d20 con 0x7f06c40d0730 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 -- 172.21.15.116:0/869622577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 msgr2=0x7f06c40cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 --2- 172.21.15.116:0/869622577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c40cbe40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f06b40097a0 tx=0x7f06b402b8d0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 -- 172.21.15.116:0/869622577 shutdown_connections 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 --2- 172.21.15.116:0/869622577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c40cbe40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 -- 172.21.15.116:0/869622577 >> 172.21.15.116:0/869622577 conn(0x7f06c40b3380 msgr2=0x7f06c40b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 -- 172.21.15.116:0/869622577 shutdown_connections 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.767+0000 7f06cc102640 1 -- 172.21.15.116:0/869622577 wait complete. 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 Processor -- start 2024-02-18T11:48:49.762 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 -- start start 2024-02-18T11:48:49.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c40b0750 0x7f06c415ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f06c4154760 0x7f06c40c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c415dd60 con 0x7f06c40d0730 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c415e040 con 0x7f06c40b0750 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06cc102640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c415e320 con 0x7f06c4154760 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06c969d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06c969d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c415d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41908/0 (socket says 172.21.15.116:41908) 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.769+0000 7f06c969d640 1 -- 172.21.15.116:0/317999936 learned_addr learned my addr 172.21.15.116:0/317999936 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06ca69f640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f06c4154760 0x7f06c40c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06c969d640 1 -- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f06c4154760 msgr2=0x7f06c40c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06c969d640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f06c4154760 0x7f06c40c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06c969d640 1 -- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c40b0750 msgr2=0x7f06c415ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06c969d640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c40b0750 0x7f06c415ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06c969d640 1 -- 172.21.15.116:0/317999936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b4009400 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06ca69f640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f06c4154760 0x7f06c40c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06c969d640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c415d510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06b40098c0 tx=0x7f06b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b403ba70 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b403bbd0 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b40453f0 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c415e600 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.770+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c415da50 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.771+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c415eb20 con 0x7f06c40d0730 2024-02-18T11:48:49.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.771+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f06b404e450 con 0x7f06c40d0730 2024-02-18T11:48:49.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.772+0000 7f06ba7fc640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f06a806bc70 0x7f06a806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:49.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.772+0000 7f06c9e9e640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f06a806bc70 0x7f06a806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:49.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.772+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f06b40acc00 con 0x7f06c40d0730 2024-02-18T11:48:49.767 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.772+0000 7f06c9e9e640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f06a806bc70 0x7f06a806e130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06bc003cc0 tx=0x7f06bc0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:49.769 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.776+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06b40b3080 con 0x7f06c40d0730 2024-02-18T11:48:49.940 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.947+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06c4074520 con 0x7f06c40d0730 2024-02-18T11:48:49.941 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.948+0000 7f06ba7fc640 1 -- 172.21.15.116:0/317999936 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16954 (secure 0 0 0) 0x7f06b4077250 con 0x7f06c40d0730 2024-02-18T11:48:49.942 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:49.942 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T11:48:48.309317+0000","last_up_change":"2024-02-18T11:47:21.411366+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:48:44.953507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-02-18T11:48:45.607007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists","up"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T11:48:49.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f06a806bc70 msgr2=0x7f06a806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f06a806bc70 0x7f06a806e130 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f06bc003cc0 tx=0x7f06bc0079b0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 msgr2=0x7f06c415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:49.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c415d510 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06b40098c0 tx=0x7f06b40046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 shutdown_connections 2024-02-18T11:48:49.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f06a806bc70 0x7f06a806e130 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f06c4154760 0x7f06c40c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f06c40d0730 0x7f06c415d510 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 --2- 172.21.15.116:0/317999936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06c40b0750 0x7f06c415ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:49.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 >> 172.21.15.116:0/317999936 conn(0x7f06c40b3380 msgr2=0x7f06c406f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:49.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 shutdown_connections 2024-02-18T11:48:49.946 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:49.950+0000 7f06cc102640 1 -- 172.21.15.116:0/317999936 wait complete. 2024-02-18T11:48:49.955 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-18T11:48:49.955 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.061+0000 7fe1b7a17640 1 Processor -- start 2024-02-18T11:48:50.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.061+0000 7fe1b7a17640 1 -- start start 2024-02-18T11:48:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b7a17640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1b00b1e50 0x7fe1b015c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b7a17640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b006d6f0 con 0x7fe1b0155e30 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b7a17640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b015c880 con 0x7fe1b015bdc0 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b7a17640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b015d690 con 0x7fe1b00b1e50 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b57b3640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe1b0155e30 0x7fe1b015cf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34746/0 (socket says 172.21.15.116:34746) 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b57b3640 1 -- 172.21.15.116:0/1392754226 learned_addr learned my addr 172.21.15.116:0/1392754226 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1b5fb4640 1 --2- 172.21.15.116:0/1392754226 >> v2:172.21.15.143:3300/0 conn(0x7fe1b00b1e50 0x7fe1b015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253538104 0 0) 0x7fe1b006d6f0 con 0x7fe1b0155e30 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe19c003520 con 0x7fe1b0155e30 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.062+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896813126 0 0) 0x7fe19c003520 con 0x7fe1b0155e30 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 >> v2:172.21.15.143:3300/0 conn(0x7fe1b00b1e50 msgr2=0x7fe1b015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 --2- 172.21.15.116:0/1392754226 >> v2:172.21.15.143:3300/0 conn(0x7fe1b00b1e50 0x7fe1b015c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 >> v1:172.21.15.143:6789/0 conn(0x7fe1b015bdc0 legacy=0x7fe1b00b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b015cb60 con 0x7fe1b0155e30 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7fe1a80019f0 con 0x7fe1b0155e30 2024-02-18T11:48:50.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 >> v1:172.21.15.116:6789/0 conn(0x7fe1b0155e30 legacy=0x7fe1b015cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe19c003970 0x7fe19c003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe19c004370 0x7fe19c004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe19c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b015d690 con 0x7fe19c005040 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c00bfa0 con 0x7fe19c003970 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c00c160 con 0x7fe19c004370 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b5fb4640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe19c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b57b3640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe19c003970 0x7fe19c003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b4fb2640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe19c004370 0x7fe19c004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b5fb4640 1 -- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe19c004370 msgr2=0x7fe19c004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b5fb4640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe19c004370 0x7fe19c004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b5fb4640 1 -- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe19c003970 msgr2=0x7fe19c003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b5fb4640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe19c003970 0x7fe19c003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.063+0000 7fe1b5fb4640 1 -- 172.21.15.116:0/1392754226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c003520 con 0x7fe19c005040 2024-02-18T11:48:50.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b5fb4640 1 -- 172.21.15.116:0/1392754226 reap_dead start 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b5fb4640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe19c00b980 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009a90 tx=0x7fe19c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3516d98209ebafe server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac004020 con 0x7fe19c005040 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac004180 con 0x7fe19c005040 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1a7fff640 1 -- 172.21.15.116:0/1392754226 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac02c6b0 con 0x7fe19c005040 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b7a17640 1 -- 172.21.15.116:0/1392754226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 msgr2=0x7fe19c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe19c00b980 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009a90 tx=0x7fe19c0057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b7a17640 1 -- 172.21.15.116:0/1392754226 shutdown_connections 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/1392754226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe19c00b980 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.064+0000 7fe1b7a17640 1 -- 172.21.15.116:0/1392754226 >> 172.21.15.116:0/1392754226 conn(0x7fe1b0066f20 msgr2=0x7fe1b006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.065+0000 7fe1b7a17640 1 -- 172.21.15.116:0/1392754226 shutdown_connections 2024-02-18T11:48:50.060 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.065+0000 7fe1b7a17640 1 -- 172.21.15.116:0/1392754226 wait complete. 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 Processor -- start 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 -- start start 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe1b00718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe1b00ab970 0x7fe1b0149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1b00a91c0 0x7fe1b00a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00ac150 con 0x7fe19c005040 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00ac430 con 0x7fe1b00a91c0 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b7a17640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b00a86e0 con 0x7fe1b00ab970 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b57b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe1b00718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.063 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.066+0000 7fe1b57b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe1b00718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41934/0 (socket says 172.21.15.116:41934) 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 -- 172.21.15.116:0/112681930 learned_addr learned my addr 172.21.15.116:0/112681930 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 -- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe1b00ab970 msgr2=0x7fe1b0149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b4fb2640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe1b00ab970 0x7fe1b0149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe1b00ab970 0x7fe1b0149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b5fb4640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1b00a91c0 0x7fe1b00a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 -- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1b00a91c0 msgr2=0x7fe1b00a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1b00a91c0 0x7fe1b00a9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 -- 172.21.15.116:0/112681930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b00a89c0 con 0x7fe19c005040 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b57b3640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe1b00718d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8007970 tx=0x7fe1a800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800ade0 con 0x7fe19c005040 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80133f0 con 0x7fe19c005040 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b00a8ca0 con 0x7fe19c005040 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8013760 con 0x7fe19c005040 2024-02-18T11:48:50.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.067+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b00abdd0 con 0x7fe19c005040 2024-02-18T11:48:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.068+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b0074520 con 0x7fe19c005040 2024-02-18T11:48:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.069+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fe1a80138c0 con 0x7fe19c005040 2024-02-18T11:48:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.069+0000 7fe1a5ffb640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe19406bca0 0x7fe19406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.069+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1a808b5b0 con 0x7fe19c005040 2024-02-18T11:48:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.069+0000 7fe1b4fb2640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe19406bca0 0x7fe19406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.065 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.069+0000 7fe1b4fb2640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe19406bca0 0x7fe19406e160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1b015aa30 tx=0x7fe1a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.066 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.073+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1a8055c00 con 0x7fe19c005040 2024-02-18T11:48:50.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.241+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1b0156210 con 0x7fe19406bca0 2024-02-18T11:48:50.241 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.247+0000 7fe1a5ffb640 1 -- 172.21.15.116:0/112681930 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248905 (secure 0 0 0) 0x7fe1a00014d0 con 0x7fe19406bca0 2024-02-18T11:48:50.244 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:50.245 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:50.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe19406bca0 msgr2=0x7fe19406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe19406bca0 0x7fe19406e160 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1b015aa30 tx=0x7fe1a0009040 comp rx=0 tx=0).stop 2024-02-18T11:48:50.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 msgr2=0x7fe1b00718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe1b00718d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8007970 tx=0x7fe1a800a730 comp rx=0 tx=0).stop 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 shutdown_connections 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fe19406bca0 0x7fe19406e160 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1b00a91c0 0x7fe1b00a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe1b00ab970 0x7fe1b0149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 --2- 172.21.15.116:0/112681930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe19c005040 0x7fe1b00718d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 >> 172.21.15.116:0/112681930 conn(0x7fe1b0066f20 msgr2=0x7fe1b015cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 shutdown_connections 2024-02-18T11:48:50.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.250+0000 7fe1b7a17640 1 -- 172.21.15.116:0/112681930 wait complete. 2024-02-18T11:48:50.254 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-02-18T11:48:48.314113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591656,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4509696,"data_stored":2598551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.008805"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319567+0000","last_change":"2024-02-18T11:48:47.319567+0000","last_active":"2024-02-18T11:48:47.319567+0000","last_peered":"2024-02-18T11:48:47.319567+0000","last_clean":"2024-02-18T11:48:47.319567+0000","last_became_active":"2024-02-18T11:48:47.318466+0000","last_became_peered":"2024-02-18T11:48:47.318466+0000","last_unstale":"2024-02-18T11:48:47.319567+0000","last_undegraded":"2024-02-18T11:48:47.319567+0000","last_fullsized":"2024-02-18T11:48:47.319567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:35:02.679276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317654+0000","last_change":"2024-02-18T11:48:47.317654+0000","last_active":"2024-02-18T11:48:47.317654+0000","last_peered":"2024-02-18T11:48:47.317654+0000","last_clean":"2024-02-18T11:48:47.317654+0000","last_became_active":"2024-02-18T11:48:47.316735+0000","last_became_peered":"2024-02-18T11:48:47.316735+0000","last_unstale":"2024-02-18T11:48:47.317654+0000","last_undegraded":"2024-02-18T11:48:47.317654+0000","last_fullsized":"2024-02-18T11:48:47.317654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305050+0000","last_change":"2024-02-18T11:48:47.305050+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305050+0000","last_undegraded":"2024-02-18T11:48:47.305050+0000","last_fullsized":"2024-02-18T11:48:47.305050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.302121+0000","last_change":"2024-02-18T11:48:47.302121+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.302121+0000","last_undegraded":"2024-02-18T11:48:47.302121+0000","last_fullsized":"2024-02-18T11:48:47.302121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301388+0000","last_change":"2024-02-18T11:48:47.301388+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301388+0000","last_undegraded":"2024-02-18T11:48:47.301388+0000","last_fullsized":"2024-02-18T11:48:47.301388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":4,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.313339+0000","last_change":"2024-02-18T11:48:47.305848+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.313339+0000","last_undegraded":"2024-02-18T11:48:47.313339+0000","last_fullsized":"2024-02-18T11:48:47.313339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304835+0000","last_change":"2024-02-18T11:48:47.304835+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304835+0000","last_undegraded":"2024-02-18T11:48:47.304835+0000","last_fullsized":"2024-02-18T11:48:47.304835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319168+0000","last_change":"2024-02-18T11:48:47.319168+0000","last_active":"2024-02-18T11:48:47.319168+0000","last_peered":"2024-02-18T11:48:47.319168+0000","last_clean":"2024-02-18T11:48:47.319168+0000","last_became_active":"2024-02-18T11:48:47.318918+0000","last_became_peered":"2024-02-18T11:48:47.318918+0000","last_unstale":"2024-02-18T11:48:47.319168+0000","last_undegraded":"2024-02-18T11:48:47.319168+0000","last_fullsized":"2024-02-18T11:48:47.319168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:03:08.526820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306155+0000","last_change":"2024-02-18T11:48:47.306155+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306155+0000","last_undegraded":"2024-02-18T11:48:47.306155+0000","last_fullsized":"2024-02-18T11:48:47.306155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317587+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.317587+0000","last_became_peered":"2024-02-18T11:48:47.317587+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305372+0000","last_change":"2024-02-18T11:48:47.305372+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305372+0000","last_undegraded":"2024-02-18T11:48:47.305372+0000","last_fullsized":"2024-02-18T11:48:47.305372+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305270+0000","last_change":"2024-02-18T11:48:47.305270+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305270+0000","last_undegraded":"2024-02-18T11:48:47.305270+0000","last_fullsized":"2024-02-18T11:48:47.305270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305144+0000","last_change":"2024-02-18T11:48:47.305144+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305144+0000","last_undegraded":"2024-02-18T11:48:47.305144+0000","last_fullsized":"2024-02-18T11:48:47.305144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320799+0000","last_change":"2024-02-18T11:48:47.314842+0000","last_active":"2024-02-18T11:48:47.320799+0000","last_peered":"2024-02-18T11:48:47.320799+0000","last_clean":"2024-02-18T11:48:47.320799+0000","last_became_active":"2024-02-18T11:48:47.312762+0000","last_became_peered":"2024-02-18T11:48:47.312762+0000","last_unstale":"2024-02-18T11:48:47.320799+0000","last_undegraded":"2024-02-18T11:48:47.320799+0000","last_fullsized":"2024-02-18T11:48:47.320799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:08.779058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304457+0000","last_change":"2024-02-18T11:48:47.304457+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304457+0000","last_undegraded":"2024-02-18T11:48:47.304457+0000","last_fullsized":"2024-02-18T11:48:47.304457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305255+0000","last_change":"2024-02-18T11:48:47.305255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305255+0000","last_undegraded":"2024-02-18T11:48:47.305255+0000","last_fullsized":"2024-02-18T11:48:47.305255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308258+0000","last_change":"2024-02-18T11:48:47.308258+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308258+0000","last_undegraded":"2024-02-18T11:48:47.308258+0000","last_fullsized":"2024-02-18T11:48:47.308258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301260+0000","last_change":"2024-02-18T11:48:47.301260+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301260+0000","last_undegraded":"2024-02-18T11:48:47.301260+0000","last_fullsized":"2024-02-18T11:48:47.301260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301207+0000","last_change":"2024-02-18T11:48:47.301207+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301207+0000","last_undegraded":"2024-02-18T11:48:47.301207+0000","last_fullsized":"2024-02-18T11:48:47.301207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305500+0000","last_change":"2024-02-18T11:48:47.305500+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305500+0000","last_undegraded":"2024-02-18T11:48:47.305500+0000","last_fullsized":"2024-02-18T11:48:47.305500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304586+0000","last_change":"2024-02-18T11:48:47.304586+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304586+0000","last_undegraded":"2024-02-18T11:48:47.304586+0000","last_fullsized":"2024-02-18T11:48:47.304586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305254+0000","last_change":"2024-02-18T11:48:47.305254+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305254+0000","last_undegraded":"2024-02-18T11:48:47.305254+0000","last_fullsized":"2024-02-18T11:48:47.305254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.300930+0000","last_change":"2024-02-18T11:48:47.300930+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.300930+0000","last_undegraded":"2024-02-18T11:48:47.300930+0000","last_fullsized":"2024-02-18T11:48:47.300930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.315231+0000","last_change":"2024-02-18T11:48:47.315173+0000","last_active":"2024-02-18T11:48:47.315231+0000","last_peered":"2024-02-18T11:48:47.315231+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.315173+0000","last_became_peered":"2024-02-18T11:48:47.315173+0000","last_unstale":"2024-02-18T11:48:47.315231+0000","last_undegraded":"2024-02-18T11:48:47.315231+0000","last_fullsized":"2024-02-18T11:48:47.315231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.310414+0000","last_change":"2024-02-18T11:48:47.310414+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.310414+0000","last_undegraded":"2024-02-18T11:48:47.310414+0000","last_fullsized":"2024-02-18T11:48:47.310414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310027+0000","last_change":"2024-02-18T11:48:47.309300+0000","last_active":"2024-02-18T11:48:47.310027+0000","last_peered":"2024-02-18T11:48:47.310027+0000","last_clean":"2024-02-18T11:48:47.310027+0000","last_became_active":"2024-02-18T11:48:47.309025+0000","last_became_peered":"2024-02-18T11:48:47.309025+0000","last_unstale":"2024-02-18T11:48:47.310027+0000","last_undegraded":"2024-02-18T11:48:47.310027+0000","last_fullsized":"2024-02-18T11:48:47.310027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306092+0000","last_change":"2024-02-18T11:48:47.306092+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306092+0000","last_undegraded":"2024-02-18T11:48:47.306092+0000","last_fullsized":"2024-02-18T11:48:47.306092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301304+0000","last_change":"2024-02-18T11:48:47.301304+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301304+0000","last_undegraded":"2024-02-18T11:48:47.301304+0000","last_fullsized":"2024-02-18T11:48:47.301304+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311897+0000","last_change":"2024-02-18T11:48:47.311897+0000","last_active":"2024-02-18T11:48:47.311897+0000","last_peered":"2024-02-18T11:48:47.311897+0000","last_clean":"2024-02-18T11:48:47.311897+0000","last_became_active":"2024-02-18T11:48:47.311571+0000","last_became_peered":"2024-02-18T11:48:47.311571+0000","last_unstale":"2024-02-18T11:48:47.311897+0000","last_undegraded":"2024-02-18T11:48:47.311897+0000","last_fullsized":"2024-02-18T11:48:47.311897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:46.613859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305634+0000","last_change":"2024-02-18T11:48:47.305634+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305634+0000","last_undegraded":"2024-02-18T11:48:47.305634+0000","last_fullsized":"2024-02-18T11:48:47.305634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305480+0000","last_change":"2024-02-18T11:48:47.305480+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305480+0000","last_undegraded":"2024-02-18T11:48:47.305480+0000","last_fullsized":"2024-02-18T11:48:47.305480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307947+0000","last_change":"2024-02-18T11:48:47.307947+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307947+0000","last_undegraded":"2024-02-18T11:48:47.307947+0000","last_fullsized":"2024-02-18T11:48:47.307947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301842+0000","last_change":"2024-02-18T11:48:46.297461+0000","last_active":"2024-02-18T11:48:47.301842+0000","last_peered":"2024-02-18T11:48:47.301842+0000","last_clean":"2024-02-18T11:48:47.301842+0000","last_became_active":"2024-02-18T11:48:46.297166+0000","last_became_peered":"2024-02-18T11:48:46.297166+0000","last_unstale":"2024-02-18T11:48:47.301842+0000","last_undegraded":"2024-02-18T11:48:47.301842+0000","last_fullsized":"2024-02-18T11:48:47.301842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306066+0000","last_change":"2024-02-18T11:48:47.306066+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306066+0000","last_undegraded":"2024-02-18T11:48:47.306066+0000","last_fullsized":"2024-02-18T11:48:47.306066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305368+0000","last_change":"2024-02-18T11:48:46.300302+0000","last_active":"2024-02-18T11:48:47.305368+0000","last_peered":"2024-02-18T11:48:47.305368+0000","last_clean":"2024-02-18T11:48:47.305368+0000","last_became_active":"2024-02-18T11:48:46.299909+0000","last_became_peered":"2024-02-18T11:48:46.299909+0000","last_unstale":"2024-02-18T11:48:47.305368+0000","last_undegraded":"2024-02-18T11:48:47.305368+0000","last_fullsized":"2024-02-18T11:48:47.305368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319795+0000","last_change":"2024-02-18T11:48:47.316579+0000","last_active":"2024-02-18T11:48:47.319795+0000","last_peered":"2024-02-18T11:48:47.319795+0000","last_clean":"2024-02-18T11:48:47.319795+0000","last_became_active":"2024-02-18T11:48:47.316195+0000","last_became_peered":"2024-02-18T11:48:47.316195+0000","last_unstale":"2024-02-18T11:48:47.319795+0000","last_undegraded":"2024-02-18T11:48:47.319795+0000","last_fullsized":"2024-02-18T11:48:47.319795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:18.861179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310566+0000","last_change":"2024-02-18T11:48:46.297756+0000","last_active":"2024-02-18T11:48:47.310566+0000","last_peered":"2024-02-18T11:48:47.310566+0000","last_clean":"2024-02-18T11:48:47.310566+0000","last_became_active":"2024-02-18T11:48:46.297485+0000","last_became_peered":"2024-02-18T11:48:46.297485+0000","last_unstale":"2024-02-18T11:48:47.310566+0000","last_undegraded":"2024-02-18T11:48:47.310566+0000","last_fullsized":"2024-02-18T11:48:47.310566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317072+0000","last_change":"2024-02-18T11:48:47.317072+0000","last_active":"2024-02-18T11:48:47.317072+0000","last_peered":"2024-02-18T11:48:47.317072+0000","last_clean":"2024-02-18T11:48:47.317072+0000","last_became_active":"2024-02-18T11:48:47.316360+0000","last_became_peered":"2024-02-18T11:48:47.316360+0000","last_unstale":"2024-02-18T11:48:47.317072+0000","last_undegraded":"2024-02-18T11:48:47.317072+0000","last_fullsized":"2024-02-18T11:48:47.317072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:08:26.674078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310420+0000","last_change":"2024-02-18T11:48:46.305539+0000","last_active":"2024-02-18T11:48:47.310420+0000","last_peered":"2024-02-18T11:48:47.310420+0000","last_clean":"2024-02-18T11:48:47.310420+0000","last_became_active":"2024-02-18T11:48:46.305252+0000","last_became_peered":"2024-02-18T11:48:46.305252+0000","last_unstale":"2024-02-18T11:48:47.310420+0000","last_undegraded":"2024-02-18T11:48:47.310420+0000","last_fullsized":"2024-02-18T11:48:47.310420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309454+0000","last_change":"2024-02-18T11:48:47.309454+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309454+0000","last_undegraded":"2024-02-18T11:48:47.309454+0000","last_fullsized":"2024-02-18T11:48:47.309454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305829+0000","last_change":"2024-02-18T11:48:46.308595+0000","last_active":"2024-02-18T11:48:47.305829+0000","last_peered":"2024-02-18T11:48:47.305829+0000","last_clean":"2024-02-18T11:48:47.305829+0000","last_became_active":"2024-02-18T11:48:46.308361+0000","last_became_peered":"2024-02-18T11:48:46.308361+0000","last_unstale":"2024-02-18T11:48:47.305829+0000","last_undegraded":"2024-02-18T11:48:47.305829+0000","last_fullsized":"2024-02-18T11:48:47.305829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305889+0000","last_change":"2024-02-18T11:48:47.305889+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305889+0000","last_undegraded":"2024-02-18T11:48:47.305889+0000","last_fullsized":"2024-02-18T11:48:47.305889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308401+0000","last_change":"2024-02-18T11:48:46.300103+0000","last_active":"2024-02-18T11:48:47.308401+0000","last_peered":"2024-02-18T11:48:47.308401+0000","last_clean":"2024-02-18T11:48:47.308401+0000","last_became_active":"2024-02-18T11:48:46.299808+0000","last_became_peered":"2024-02-18T11:48:46.299808+0000","last_unstale":"2024-02-18T11:48:47.308401+0000","last_undegraded":"2024-02-18T11:48:47.308401+0000","last_fullsized":"2024-02-18T11:48:47.308401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320639+0000","last_change":"2024-02-18T11:48:47.320639+0000","last_active":"2024-02-18T11:48:47.320639+0000","last_peered":"2024-02-18T11:48:47.320639+0000","last_clean":"2024-02-18T11:48:47.320639+0000","last_became_active":"2024-02-18T11:48:47.320385+0000","last_became_peered":"2024-02-18T11:48:47.320385+0000","last_unstale":"2024-02-18T11:48:47.320639+0000","last_undegraded":"2024-02-18T11:48:47.320639+0000","last_fullsized":"2024-02-18T11:48:47.320639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:57.144454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310023+0000","last_change":"2024-02-18T11:48:46.304701+0000","last_active":"2024-02-18T11:48:47.310023+0000","last_peered":"2024-02-18T11:48:47.310023+0000","last_clean":"2024-02-18T11:48:47.310023+0000","last_became_active":"2024-02-18T11:48:46.304450+0000","last_became_peered":"2024-02-18T11:48:46.304450+0000","last_unstale":"2024-02-18T11:48:47.310023+0000","last_undegraded":"2024-02-18T11:48:47.310023+0000","last_fullsized":"2024-02-18T11:48:47.310023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317646+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:47.317646+0000","last_became_active":"2024-02-18T11:48:47.317171+0000","last_became_peered":"2024-02-18T11:48:47.317171+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:14.550912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304637+0000","last_change":"2024-02-18T11:48:46.296450+0000","last_active":"2024-02-18T11:48:47.304637+0000","last_peered":"2024-02-18T11:48:47.304637+0000","last_clean":"2024-02-18T11:48:47.304637+0000","last_became_active":"2024-02-18T11:48:46.296159+0000","last_became_peered":"2024-02-18T11:48:46.296159+0000","last_unstale":"2024-02-18T11:48:47.304637+0000","last_undegraded":"2024-02-18T11:48:47.304637+0000","last_fullsized":"2024-02-18T11:48:47.304637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303802+0000","last_change":"2024-02-18T11:48:46.306411+0000","last_active":"2024-02-18T11:48:47.303802+0000","last_peered":"2024-02-18T11:48:47.303802+0000","last_clean":"2024-02-18T11:48:47.303802+0000","last_became_active":"2024-02-18T11:48:46.304718+0000","last_became_peered":"2024-02-18T11:48:46.304718+0000","last_unstale":"2024-02-18T11:48:47.303802+0000","last_undegraded":"2024-02-18T11:48:47.303802+0000","last_fullsized":"2024-02-18T11:48:47.303802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311877+0000","last_change":"2024-02-18T11:48:47.311877+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311877+0000","last_undegraded":"2024-02-18T11:48:47.311877+0000","last_fullsized":"2024-02-18T11:48:47.311877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308363+0000","last_change":"2024-02-18T11:48:46.307561+0000","last_active":"2024-02-18T11:48:47.308363+0000","last_peered":"2024-02-18T11:48:47.308363+0000","last_clean":"2024-02-18T11:48:47.308363+0000","last_became_active":"2024-02-18T11:48:46.307253+0000","last_became_peered":"2024-02-18T11:48:46.307253+0000","last_unstale":"2024-02-18T11:48:47.308363+0000","last_undegraded":"2024-02-18T11:48:47.308363+0000","last_fullsized":"2024-02-18T11:48:47.308363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.316738+0000","last_change":"2024-02-18T11:48:47.316738+0000","last_active":"2024-02-18T11:48:47.316738+0000","last_peered":"2024-02-18T11:48:47.316738+0000","last_clean":"2024-02-18T11:48:47.316738+0000","last_became_active":"2024-02-18T11:48:47.316488+0000","last_became_peered":"2024-02-18T11:48:47.316488+0000","last_unstale":"2024-02-18T11:48:47.316738+0000","last_undegraded":"2024-02-18T11:48:47.316738+0000","last_fullsized":"2024-02-18T11:48:47.316738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305690+0000","last_change":"2024-02-18T11:48:46.304153+0000","last_active":"2024-02-18T11:48:47.305690+0000","last_peered":"2024-02-18T11:48:47.305690+0000","last_clean":"2024-02-18T11:48:47.305690+0000","last_became_active":"2024-02-18T11:48:46.303687+0000","last_became_peered":"2024-02-18T11:48:46.303687+0000","last_unstale":"2024-02-18T11:48:47.305690+0000","last_undegraded":"2024-02-18T11:48:47.305690+0000","last_fullsized":"2024-02-18T11:48:47.305690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307009+0000","last_change":"2024-02-18T11:48:47.307009+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307009+0000","last_undegraded":"2024-02-18T11:48:47.307009+0000","last_fullsized":"2024-02-18T11:48:47.307009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304472+0000","last_change":"2024-02-18T11:48:46.301330+0000","last_active":"2024-02-18T11:48:47.304472+0000","last_peered":"2024-02-18T11:48:47.304472+0000","last_clean":"2024-02-18T11:48:47.304472+0000","last_became_active":"2024-02-18T11:48:46.301062+0000","last_became_peered":"2024-02-18T11:48:46.301062+0000","last_unstale":"2024-02-18T11:48:47.304472+0000","last_undegraded":"2024-02-18T11:48:47.304472+0000","last_fullsized":"2024-02-18T11:48:47.304472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309147+0000","last_change":"2024-02-18T11:48:47.309147+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309147+0000","last_undegraded":"2024-02-18T11:48:47.309147+0000","last_fullsized":"2024-02-18T11:48:47.309147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304790+0000","last_change":"2024-02-18T11:48:46.307286+0000","last_active":"2024-02-18T11:48:47.304790+0000","last_peered":"2024-02-18T11:48:47.304790+0000","last_clean":"2024-02-18T11:48:47.304790+0000","last_became_active":"2024-02-18T11:48:46.307002+0000","last_became_peered":"2024-02-18T11:48:46.307002+0000","last_unstale":"2024-02-18T11:48:47.304790+0000","last_undegraded":"2024-02-18T11:48:47.304790+0000","last_fullsized":"2024-02-18T11:48:47.304790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:21.137008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323364+0000","last_change":"2024-02-18T11:48:47.323364+0000","last_active":"2024-02-18T11:48:47.323364+0000","last_peered":"2024-02-18T11:48:47.323364+0000","last_clean":"2024-02-18T11:48:47.323364+0000","last_became_active":"2024-02-18T11:48:47.318734+0000","last_became_peered":"2024-02-18T11:48:47.318734+0000","last_unstale":"2024-02-18T11:48:47.323364+0000","last_undegraded":"2024-02-18T11:48:47.323364+0000","last_fullsized":"2024-02-18T11:48:47.323364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:29:29.908143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306130+0000","last_change":"2024-02-18T11:48:46.305434+0000","last_active":"2024-02-18T11:48:47.306130+0000","last_peered":"2024-02-18T11:48:47.306130+0000","last_clean":"2024-02-18T11:48:47.306130+0000","last_became_active":"2024-02-18T11:48:46.305145+0000","last_became_peered":"2024-02-18T11:48:46.305145+0000","last_unstale":"2024-02-18T11:48:47.306130+0000","last_undegraded":"2024-02-18T11:48:47.306130+0000","last_fullsized":"2024-02-18T11:48:47.306130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:07.775308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308255+0000","last_change":"2024-02-18T11:48:47.308255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308255+0000","last_undegraded":"2024-02-18T11:48:47.308255+0000","last_fullsized":"2024-02-18T11:48:47.308255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301908+0000","last_change":"2024-02-18T11:48:46.297664+0000","last_active":"2024-02-18T11:48:47.301908+0000","last_peered":"2024-02-18T11:48:47.301908+0000","last_clean":"2024-02-18T11:48:47.301908+0000","last_became_active":"2024-02-18T11:48:46.297202+0000","last_became_peered":"2024-02-18T11:48:46.297202+0000","last_unstale":"2024-02-18T11:48:47.301908+0000","last_undegraded":"2024-02-18T11:48:47.301908+0000","last_fullsized":"2024-02-18T11:48:47.301908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.318448+0000","last_change":"2024-02-18T11:48:47.318448+0000","last_active":"2024-02-18T11:48:47.318448+0000","last_peered":"2024-02-18T11:48:47.318448+0000","last_clean":"2024-02-18T11:48:47.318448+0000","last_became_active":"2024-02-18T11:48:47.318190+0000","last_became_peered":"2024-02-18T11:48:47.318190+0000","last_unstale":"2024-02-18T11:48:47.318448+0000","last_undegraded":"2024-02-18T11:48:47.318448+0000","last_fullsized":"2024-02-18T11:48:47.318448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:15:45.332480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306501+0000","last_change":"2024-02-18T11:48:46.300667+0000","last_active":"2024-02-18T11:48:47.306501+0000","last_peered":"2024-02-18T11:48:47.306501+0000","last_clean":"2024-02-18T11:48:47.306501+0000","last_became_active":"2024-02-18T11:48:46.300407+0000","last_became_peered":"2024-02-18T11:48:46.300407+0000","last_unstale":"2024-02-18T11:48:47.306501+0000","last_undegraded":"2024-02-18T11:48:47.306501+0000","last_fullsized":"2024-02-18T11:48:47.306501+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323994+0000","last_change":"2024-02-18T11:48:47.309612+0000","last_active":"2024-02-18T11:48:47.323994+0000","last_peered":"2024-02-18T11:48:47.323994+0000","last_clean":"2024-02-18T11:48:47.323994+0000","last_became_active":"2024-02-18T11:48:47.309382+0000","last_became_peered":"2024-02-18T11:48:47.309382+0000","last_unstale":"2024-02-18T11:48:47.323994+0000","last_undegraded":"2024-02-18T11:48:47.323994+0000","last_fullsized":"2024-02-18T11:48:47.323994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311657+0000","last_change":"2024-02-18T11:48:46.307646+0000","last_active":"2024-02-18T11:48:47.311657+0000","last_peered":"2024-02-18T11:48:47.311657+0000","last_clean":"2024-02-18T11:48:47.311657+0000","last_became_active":"2024-02-18T11:48:46.307345+0000","last_became_peered":"2024-02-18T11:48:46.307345+0000","last_unstale":"2024-02-18T11:48:47.311657+0000","last_undegraded":"2024-02-18T11:48:47.311657+0000","last_fullsized":"2024-02-18T11:48:47.311657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:50.554971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304896+0000","last_change":"2024-02-18T11:48:47.304896+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304896+0000","last_undegraded":"2024-02-18T11:48:47.304896+0000","last_fullsized":"2024-02-18T11:48:47.304896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306767+0000","last_change":"2024-02-18T11:48:46.306743+0000","last_active":"2024-02-18T11:48:47.306767+0000","last_peered":"2024-02-18T11:48:47.306767+0000","last_clean":"2024-02-18T11:48:47.306767+0000","last_became_active":"2024-02-18T11:48:46.306447+0000","last_became_peered":"2024-02-18T11:48:46.306447+0000","last_unstale":"2024-02-18T11:48:47.306767+0000","last_undegraded":"2024-02-18T11:48:47.306767+0000","last_fullsized":"2024-02-18T11:48:47.306767+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.321271+0000","last_change":"2024-02-18T11:48:47.321271+0000","last_active":"2024-02-18T11:48:47.321271+0000","last_peered":"2024-02-18T11:48:47.321271+0000","last_clean":"2024-02-18T11:48:47.321271+0000","last_became_active":"2024-02-18T11:48:47.321021+0000","last_became_peered":"2024-02-18T11:48:47.321021+0000","last_unstale":"2024-02-18T11:48:47.321271+0000","last_undegraded":"2024-02-18T11:48:47.321271+0000","last_fullsized":"2024-02-18T11:48:47.321271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301977+0000","last_change":"2024-02-18T11:48:46.296023+0000","last_active":"2024-02-18T11:48:47.301977+0000","last_peered":"2024-02-18T11:48:47.301977+0000","last_clean":"2024-02-18T11:48:47.301977+0000","last_became_active":"2024-02-18T11:48:46.295715+0000","last_became_peered":"2024-02-18T11:48:46.295715+0000","last_unstale":"2024-02-18T11:48:47.301977+0000","last_undegraded":"2024-02-18T11:48:47.301977+0000","last_fullsized":"2024-02-18T11:48:47.301977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310131+0000","last_change":"2024-02-18T11:48:47.310131+0000","last_active":"2024-02-18T11:48:47.310131+0000","last_peered":"2024-02-18T11:48:47.310131+0000","last_clean":"2024-02-18T11:48:47.310131+0000","last_became_active":"2024-02-18T11:48:47.309877+0000","last_became_peered":"2024-02-18T11:48:47.309877+0000","last_unstale":"2024-02-18T11:48:47.310131+0000","last_undegraded":"2024-02-18T11:48:47.310131+0000","last_fullsized":"2024-02-18T11:48:47.310131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305646+0000","last_change":"2024-02-18T11:48:46.297720+0000","last_active":"2024-02-18T11:48:47.305646+0000","last_peered":"2024-02-18T11:48:47.305646+0000","last_clean":"2024-02-18T11:48:47.305646+0000","last_became_active":"2024-02-18T11:48:46.297462+0000","last_became_peered":"2024-02-18T11:48:46.297462+0000","last_unstale":"2024-02-18T11:48:47.305646+0000","last_undegraded":"2024-02-18T11:48:47.305646+0000","last_fullsized":"2024-02-18T11:48:47.305646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306864+0000","last_change":"2024-02-18T11:48:47.306864+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306864+0000","last_undegraded":"2024-02-18T11:48:47.306864+0000","last_fullsized":"2024-02-18T11:48:47.306864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301800+0000","last_change":"2024-02-18T11:48:46.303136+0000","last_active":"2024-02-18T11:48:47.301800+0000","last_peered":"2024-02-18T11:48:47.301800+0000","last_clean":"2024-02-18T11:48:47.301800+0000","last_became_active":"2024-02-18T11:48:46.302861+0000","last_became_peered":"2024-02-18T11:48:46.302861+0000","last_unstale":"2024-02-18T11:48:47.301800+0000","last_undegraded":"2024-02-18T11:48:47.301800+0000","last_fullsized":"2024-02-18T11:48:47.301800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306944+0000","last_change":"2024-02-18T11:48:47.306944+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306944+0000","last_undegraded":"2024-02-18T11:48:47.306944+0000","last_fullsized":"2024-02-18T11:48:47.306944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.300950+0000","last_change":"2024-02-18T11:48:46.297761+0000","last_active":"2024-02-18T11:48:47.300950+0000","last_peered":"2024-02-18T11:48:47.300950+0000","last_clean":"2024-02-18T11:48:47.300950+0000","last_became_active":"2024-02-18T11:48:46.297500+0000","last_became_peered":"2024-02-18T11:48:46.297500+0000","last_unstale":"2024-02-18T11:48:47.300950+0000","last_undegraded":"2024-02-18T11:48:47.300950+0000","last_fullsized":"2024-02-18T11:48:47.300950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:33:10.032538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310081+0000","last_change":"2024-02-18T11:48:47.310081+0000","last_active":"2024-02-18T11:48:47.310081+0000","last_peered":"2024-02-18T11:48:47.310081+0000","last_clean":"2024-02-18T11:48:47.310081+0000","last_became_active":"2024-02-18T11:48:47.309813+0000","last_became_peered":"2024-02-18T11:48:47.309813+0000","last_unstale":"2024-02-18T11:48:47.310081+0000","last_undegraded":"2024-02-18T11:48:47.310081+0000","last_fullsized":"2024-02-18T11:48:47.310081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306548+0000","last_change":"2024-02-18T11:48:46.298569+0000","last_active":"2024-02-18T11:48:47.306548+0000","last_peered":"2024-02-18T11:48:47.306548+0000","last_clean":"2024-02-18T11:48:47.306548+0000","last_became_active":"2024-02-18T11:48:46.297865+0000","last_became_peered":"2024-02-18T11:48:46.297865+0000","last_unstale":"2024-02-18T11:48:47.306548+0000","last_undegraded":"2024-02-18T11:48:47.306548+0000","last_fullsized":"2024-02-18T11:48:47.306548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:04.067605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.312852+0000","last_change":"2024-02-18T11:48:47.312852+0000","last_active":"2024-02-18T11:48:47.312852+0000","last_peered":"2024-02-18T11:48:47.312852+0000","last_clean":"2024-02-18T11:48:47.312852+0000","last_became_active":"2024-02-18T11:48:47.312584+0000","last_became_peered":"2024-02-18T11:48:47.312584+0000","last_unstale":"2024-02-18T11:48:47.312852+0000","last_undegraded":"2024-02-18T11:48:47.312852+0000","last_fullsized":"2024-02-18T11:48:47.312852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:51.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301208+0000","last_change":"2024-02-18T11:48:46.302429+0000","last_active":"2024-02-18T11:48:47.301208+0000","last_peered":"2024-02-18T11:48:47.301208+0000","last_clean":"2024-02-18T11:48:47.301208+0000","last_became_active":"2024-02-18T11:48:46.302138+0000","last_became_peered":"2024-02-18T11:48:46.302138+0000","last_unstale":"2024-02-18T11:48:47.301208+0000","last_undegraded":"2024-02-18T11:48:47.301208+0000","last_fullsized":"2024-02-18T11:48:47.301208+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:54.091351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305364+0000","last_change":"2024-02-18T11:48:47.305364+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305364+0000","last_undegraded":"2024-02-18T11:48:47.305364+0000","last_fullsized":"2024-02-18T11:48:47.305364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311359+0000","last_change":"2024-02-18T11:48:46.308271+0000","last_active":"2024-02-18T11:48:47.311359+0000","last_peered":"2024-02-18T11:48:47.311359+0000","last_clean":"2024-02-18T11:48:47.311359+0000","last_became_active":"2024-02-18T11:48:46.307981+0000","last_became_peered":"2024-02-18T11:48:46.307981+0000","last_unstale":"2024-02-18T11:48:47.311359+0000","last_undegraded":"2024-02-18T11:48:47.311359+0000","last_fullsized":"2024-02-18T11:48:47.311359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:32.990774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311673+0000","last_change":"2024-02-18T11:48:47.311673+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311673+0000","last_undegraded":"2024-02-18T11:48:47.311673+0000","last_fullsized":"2024-02-18T11:48:47.311673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301714+0000","last_change":"2024-02-18T11:48:46.305551+0000","last_active":"2024-02-18T11:48:47.301714+0000","last_peered":"2024-02-18T11:48:47.301714+0000","last_clean":"2024-02-18T11:48:47.301714+0000","last_became_active":"2024-02-18T11:48:46.305246+0000","last_became_peered":"2024-02-18T11:48:46.305246+0000","last_unstale":"2024-02-18T11:48:47.301714+0000","last_undegraded":"2024-02-18T11:48:47.301714+0000","last_fullsized":"2024-02-18T11:48:47.301714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319083+0000","last_change":"2024-02-18T11:48:47.319083+0000","last_active":"2024-02-18T11:48:47.319083+0000","last_peered":"2024-02-18T11:48:47.319083+0000","last_clean":"2024-02-18T11:48:47.319083+0000","last_became_active":"2024-02-18T11:48:47.318762+0000","last_became_peered":"2024-02-18T11:48:47.318762+0000","last_unstale":"2024-02-18T11:48:47.319083+0000","last_undegraded":"2024-02-18T11:48:47.319083+0000","last_fullsized":"2024-02-18T11:48:47.319083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:55:35.888377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305354+0000","last_change":"2024-02-18T11:48:46.305494+0000","last_active":"2024-02-18T11:48:47.305354+0000","last_peered":"2024-02-18T11:48:47.305354+0000","last_clean":"2024-02-18T11:48:47.305354+0000","last_became_active":"2024-02-18T11:48:46.305222+0000","last_became_peered":"2024-02-18T11:48:46.305222+0000","last_unstale":"2024-02-18T11:48:47.305354+0000","last_undegraded":"2024-02-18T11:48:47.305354+0000","last_fullsized":"2024-02-18T11:48:47.305354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308833+0000","last_change":"2024-02-18T11:48:47.308833+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308833+0000","last_undegraded":"2024-02-18T11:48:47.308833+0000","last_fullsized":"2024-02-18T11:48:47.308833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308558+0000","last_change":"2024-02-18T11:48:46.303722+0000","last_active":"2024-02-18T11:48:47.308558+0000","last_peered":"2024-02-18T11:48:47.308558+0000","last_clean":"2024-02-18T11:48:47.308558+0000","last_became_active":"2024-02-18T11:48:46.303460+0000","last_became_peered":"2024-02-18T11:48:46.303460+0000","last_unstale":"2024-02-18T11:48:47.308558+0000","last_undegraded":"2024-02-18T11:48:47.308558+0000","last_fullsized":"2024-02-18T11:48:47.308558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309759+0000","last_change":"2024-02-18T11:48:47.309759+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309759+0000","last_undegraded":"2024-02-18T11:48:47.309759+0000","last_fullsized":"2024-02-18T11:48:47.309759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.302363+0000","last_change":"2024-02-18T11:48:46.299505+0000","last_active":"2024-02-18T11:48:47.302363+0000","last_peered":"2024-02-18T11:48:47.302363+0000","last_clean":"2024-02-18T11:48:47.302363+0000","last_became_active":"2024-02-18T11:48:46.299230+0000","last_became_peered":"2024-02-18T11:48:46.299230+0000","last_unstale":"2024-02-18T11:48:47.302363+0000","last_undegraded":"2024-02-18T11:48:47.302363+0000","last_fullsized":"2024-02-18T11:48:47.302363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320820+0000","last_change":"2024-02-18T11:48:47.315114+0000","last_active":"2024-02-18T11:48:47.320820+0000","last_peered":"2024-02-18T11:48:47.320820+0000","last_clean":"2024-02-18T11:48:47.320820+0000","last_became_active":"2024-02-18T11:48:47.313064+0000","last_became_peered":"2024-02-18T11:48:47.313064+0000","last_unstale":"2024-02-18T11:48:47.320820+0000","last_undegraded":"2024-02-18T11:48:47.320820+0000","last_fullsized":"2024-02-18T11:48:47.320820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:22.773038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307959+0000","last_change":"2024-02-18T11:48:46.306424+0000","last_active":"2024-02-18T11:48:47.307959+0000","last_peered":"2024-02-18T11:48:47.307959+0000","last_clean":"2024-02-18T11:48:47.307959+0000","last_became_active":"2024-02-18T11:48:46.305938+0000","last_became_peered":"2024-02-18T11:48:46.305938+0000","last_unstale":"2024-02-18T11:48:47.307959+0000","last_undegraded":"2024-02-18T11:48:47.307959+0000","last_fullsized":"2024-02-18T11:48:47.307959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:48.967965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.314001+0000","last_change":"2024-02-18T11:48:47.314001+0000","last_active":"2024-02-18T11:48:47.314001+0000","last_peered":"2024-02-18T11:48:47.314001+0000","last_clean":"2024-02-18T11:48:47.314001+0000","last_became_active":"2024-02-18T11:48:47.313680+0000","last_became_peered":"2024-02-18T11:48:47.313680+0000","last_unstale":"2024-02-18T11:48:47.314001+0000","last_undegraded":"2024-02-18T11:48:47.314001+0000","last_fullsized":"2024-02-18T11:48:47.314001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303228+0000","last_change":"2024-02-18T11:48:46.306107+0000","last_active":"2024-02-18T11:48:47.303228+0000","last_peered":"2024-02-18T11:48:47.303228+0000","last_clean":"2024-02-18T11:48:47.303228+0000","last_became_active":"2024-02-18T11:48:46.305749+0000","last_became_peered":"2024-02-18T11:48:46.305749+0000","last_unstale":"2024-02-18T11:48:47.303228+0000","last_undegraded":"2024-02-18T11:48:47.303228+0000","last_fullsized":"2024-02-18T11:48:47.303228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311076+0000","last_change":"2024-02-18T11:48:47.311076+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311076+0000","last_undegraded":"2024-02-18T11:48:47.311076+0000","last_fullsized":"2024-02-18T11:48:47.311076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307802+0000","last_change":"2024-02-18T11:48:46.303447+0000","last_active":"2024-02-18T11:48:47.307802+0000","last_peered":"2024-02-18T11:48:47.307802+0000","last_clean":"2024-02-18T11:48:47.307802+0000","last_became_active":"2024-02-18T11:48:46.303126+0000","last_became_peered":"2024-02-18T11:48:46.303126+0000","last_unstale":"2024-02-18T11:48:47.307802+0000","last_undegraded":"2024-02-18T11:48:47.307802+0000","last_fullsized":"2024-02-18T11:48:47.307802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:44.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307052+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:48:47.307052+0000","last_peered":"2024-02-18T11:48:47.307052+0000","last_clean":"2024-02-18T11:48:47.307052+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:48:47.307052+0000","last_undegraded":"2024-02-18T11:48:47.307052+0000","last_fullsized":"2024-02-18T11:48:47.307052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21,"ondisk_log_size":21,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:50.259 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:50.396 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.403+0000 7fa8ee6c0640 1 Processor -- start 2024-02-18T11:48:50.397 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.403+0000 7fa8ee6c0640 1 -- start start 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ee6c0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e8155e30 0x7fa8e815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ee6c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8e815bdc0 0x7fa8e815c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ee6c0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8e80b1740 0x7fa8e80b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ee6c0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80b2110 con 0x7fa8e815bdc0 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ee6c0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e806d6f0 con 0x7fa8e80b1740 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ee6c0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815c880 con 0x7fa8e8155e30 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8e815bdc0 0x7fa8e815c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8e815bdc0 0x7fa8e815c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41950/0 (socket says 172.21.15.116:41950) 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 -- 172.21.15.116:0/3285150678 learned_addr learned my addr 172.21.15.116:0/3285150678 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e7fff640 1 --2- 172.21.15.116:0/3285150678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e8155e30 0x7fa8e815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.398 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8ecc5d640 1 --2- 172.21.15.116:0/3285150678 >> v2:172.21.15.143:3300/0 conn(0x7fa8e80b1740 0x7fa8e80b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 -- 172.21.15.116:0/3285150678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e8155e30 msgr2=0x7fa8e815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 --2- 172.21.15.116:0/3285150678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e8155e30 0x7fa8e815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 -- 172.21.15.116:0/3285150678 >> v2:172.21.15.143:3300/0 conn(0x7fa8e80b1740 msgr2=0x7fa8e80b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 --2- 172.21.15.116:0/3285150678 >> v2:172.21.15.143:3300/0 conn(0x7fa8e80b1740 0x7fa8e80b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.399 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.404+0000 7fa8e77fe640 1 -- 172.21.15.116:0/3285150678 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e815cb60 con 0x7fa8e815bdc0 2024-02-18T11:48:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.405+0000 7fa8e77fe640 1 --2- 172.21.15.116:0/3285150678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e815c240 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa8e815c780 tx=0x7fa8dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03a82cce1db7837 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.405+0000 7fa8e6ffd640 1 -- 172.21.15.116:0/3285150678 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc031870 con 0x7fa8e815bdc0 2024-02-18T11:48:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.405+0000 7fa8e6ffd640 1 -- 172.21.15.116:0/3285150678 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8dc0319d0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.400 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.405+0000 7fa8e6ffd640 1 -- 172.21.15.116:0/3285150678 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc031d60 con 0x7fa8e815bdc0 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/3285150678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 msgr2=0x7fa8e815c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/3285150678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e815c240 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa8e815c780 tx=0x7fa8dc02b910 comp rx=0 tx=0).stop 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/3285150678 shutdown_connections 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/3285150678 >> v2:172.21.15.143:3300/0 conn(0x7fa8e80b1740 0x7fa8e80b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/3285150678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e815c240 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/3285150678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e8155e30 0x7fa8e815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/3285150678 >> 172.21.15.116:0/3285150678 conn(0x7fa8e8066f20 msgr2=0x7fa8e806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/3285150678 shutdown_connections 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.406+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/3285150678 wait complete. 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.407+0000 7fa8ee6c0640 1 Processor -- start 2024-02-18T11:48:50.401 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.407+0000 7fa8ee6c0640 1 -- start start 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ee6c0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e80b1740 0x7fa8e81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ee6c0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8e8155e30 0x7fa8e80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ee6c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e80b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ee6c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81527f0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ee6c0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80afc20 con 0x7fa8e8155e30 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ee6c0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80afef0 con 0x7fa8e80b1740 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e80b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e80b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41958/0 (socket says 172.21.15.116:41958) 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 -- 172.21.15.116:0/2297175566 learned_addr learned my addr 172.21.15.116:0/2297175566 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8e7fff640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e80b1740 0x7fa8e81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8e77fe640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8e8155e30 0x7fa8e80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 -- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e80b1740 msgr2=0x7fa8e81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e80b1740 0x7fa8e81522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 -- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8e8155e30 msgr2=0x7fa8e80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8e8155e30 0x7fa8e80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8ecc5d640 1 -- 172.21.15.116:0/2297175566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc0094d0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.408+0000 7fa8e7fff640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e80b1740 0x7fa8e81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.409+0000 7fa8ecc5d640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e80b0f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0007e60 tx=0x7fa8e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.409+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0014620 con 0x7fa8e815bdc0 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.409+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e000ee90 con 0x7fa8e815bdc0 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.409+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e80b01d0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.409+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00149e0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.409+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e80b0790 con 0x7fa8e815bdc0 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.410+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e8074520 con 0x7fa8e815bdc0 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.410+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa8e0014b40 con 0x7fa8e815bdc0 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.410+0000 7fa8e4ff9640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8d406bc70 0x7fa8d406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.411+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa8e008b1e0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.411+0000 7fa8e7fff640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8d406bc70 0x7fa8d406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.406 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.411+0000 7fa8e7fff640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8d406bc70 0x7fa8d406e130 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.415+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8e00558c0 con 0x7fa8e815bdc0 2024-02-18T11:48:50.574 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.581+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8e806b830 con 0x7fa8d406bc70 2024-02-18T11:48:50.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.587+0000 7fa8e4ff9640 1 -- 172.21.15.116:0/2297175566 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248905 (secure 0 0 0) 0x7fa8e806b830 con 0x7fa8d406bc70 2024-02-18T11:48:50.583 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:50.584 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8d406bc70 msgr2=0x7fa8d406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8d406bc70 0x7fa8d406e130 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).stop 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 msgr2=0x7fa8e80b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e80b0f90 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0007e60 tx=0x7fa8e000e4b0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 shutdown_connections 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa8d406bc70 0x7fa8d406e130 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8e815bdc0 0x7fa8e80b0f90 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8e8155e30 0x7fa8e80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 --2- 172.21.15.116:0/2297175566 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8e80b1740 0x7fa8e81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 >> 172.21.15.116:0/2297175566 conn(0x7fa8e8066f20 msgr2=0x7fa8e80b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 shutdown_connections 2024-02-18T11:48:50.586 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.590+0000 7fa8ee6c0640 1 -- 172.21.15.116:0/2297175566 wait complete. 2024-02-18T11:48:50.594 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-02-18T11:48:50.306500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591656,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4509696,"data_stored":2598551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.024513"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319567+0000","last_change":"2024-02-18T11:48:47.319567+0000","last_active":"2024-02-18T11:48:47.319567+0000","last_peered":"2024-02-18T11:48:47.319567+0000","last_clean":"2024-02-18T11:48:47.319567+0000","last_became_active":"2024-02-18T11:48:47.318466+0000","last_became_peered":"2024-02-18T11:48:47.318466+0000","last_unstale":"2024-02-18T11:48:47.319567+0000","last_undegraded":"2024-02-18T11:48:47.319567+0000","last_fullsized":"2024-02-18T11:48:47.319567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:35:02.679276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317654+0000","last_change":"2024-02-18T11:48:47.317654+0000","last_active":"2024-02-18T11:48:47.317654+0000","last_peered":"2024-02-18T11:48:47.317654+0000","last_clean":"2024-02-18T11:48:47.317654+0000","last_became_active":"2024-02-18T11:48:47.316735+0000","last_became_peered":"2024-02-18T11:48:47.316735+0000","last_unstale":"2024-02-18T11:48:47.317654+0000","last_undegraded":"2024-02-18T11:48:47.317654+0000","last_fullsized":"2024-02-18T11:48:47.317654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305050+0000","last_change":"2024-02-18T11:48:47.305050+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305050+0000","last_undegraded":"2024-02-18T11:48:47.305050+0000","last_fullsized":"2024-02-18T11:48:47.305050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.302121+0000","last_change":"2024-02-18T11:48:47.302121+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.302121+0000","last_undegraded":"2024-02-18T11:48:47.302121+0000","last_fullsized":"2024-02-18T11:48:47.302121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301388+0000","last_change":"2024-02-18T11:48:47.301388+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301388+0000","last_undegraded":"2024-02-18T11:48:47.301388+0000","last_fullsized":"2024-02-18T11:48:47.301388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":4,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.313339+0000","last_change":"2024-02-18T11:48:47.305848+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.313339+0000","last_undegraded":"2024-02-18T11:48:47.313339+0000","last_fullsized":"2024-02-18T11:48:47.313339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304835+0000","last_change":"2024-02-18T11:48:47.304835+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304835+0000","last_undegraded":"2024-02-18T11:48:47.304835+0000","last_fullsized":"2024-02-18T11:48:47.304835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319168+0000","last_change":"2024-02-18T11:48:47.319168+0000","last_active":"2024-02-18T11:48:47.319168+0000","last_peered":"2024-02-18T11:48:47.319168+0000","last_clean":"2024-02-18T11:48:47.319168+0000","last_became_active":"2024-02-18T11:48:47.318918+0000","last_became_peered":"2024-02-18T11:48:47.318918+0000","last_unstale":"2024-02-18T11:48:47.319168+0000","last_undegraded":"2024-02-18T11:48:47.319168+0000","last_fullsized":"2024-02-18T11:48:47.319168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:03:08.526820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306155+0000","last_change":"2024-02-18T11:48:47.306155+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306155+0000","last_undegraded":"2024-02-18T11:48:47.306155+0000","last_fullsized":"2024-02-18T11:48:47.306155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317587+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.317587+0000","last_became_peered":"2024-02-18T11:48:47.317587+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305372+0000","last_change":"2024-02-18T11:48:47.305372+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305372+0000","last_undegraded":"2024-02-18T11:48:47.305372+0000","last_fullsized":"2024-02-18T11:48:47.305372+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305270+0000","last_change":"2024-02-18T11:48:47.305270+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305270+0000","last_undegraded":"2024-02-18T11:48:47.305270+0000","last_fullsized":"2024-02-18T11:48:47.305270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305144+0000","last_change":"2024-02-18T11:48:47.305144+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305144+0000","last_undegraded":"2024-02-18T11:48:47.305144+0000","last_fullsized":"2024-02-18T11:48:47.305144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320799+0000","last_change":"2024-02-18T11:48:47.314842+0000","last_active":"2024-02-18T11:48:47.320799+0000","last_peered":"2024-02-18T11:48:47.320799+0000","last_clean":"2024-02-18T11:48:47.320799+0000","last_became_active":"2024-02-18T11:48:47.312762+0000","last_became_peered":"2024-02-18T11:48:47.312762+0000","last_unstale":"2024-02-18T11:48:47.320799+0000","last_undegraded":"2024-02-18T11:48:47.320799+0000","last_fullsized":"2024-02-18T11:48:47.320799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:08.779058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304457+0000","last_change":"2024-02-18T11:48:47.304457+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304457+0000","last_undegraded":"2024-02-18T11:48:47.304457+0000","last_fullsized":"2024-02-18T11:48:47.304457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305255+0000","last_change":"2024-02-18T11:48:47.305255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305255+0000","last_undegraded":"2024-02-18T11:48:47.305255+0000","last_fullsized":"2024-02-18T11:48:47.305255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308258+0000","last_change":"2024-02-18T11:48:47.308258+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308258+0000","last_undegraded":"2024-02-18T11:48:47.308258+0000","last_fullsized":"2024-02-18T11:48:47.308258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301260+0000","last_change":"2024-02-18T11:48:47.301260+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301260+0000","last_undegraded":"2024-02-18T11:48:47.301260+0000","last_fullsized":"2024-02-18T11:48:47.301260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301207+0000","last_change":"2024-02-18T11:48:47.301207+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301207+0000","last_undegraded":"2024-02-18T11:48:47.301207+0000","last_fullsized":"2024-02-18T11:48:47.301207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305500+0000","last_change":"2024-02-18T11:48:47.305500+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305500+0000","last_undegraded":"2024-02-18T11:48:47.305500+0000","last_fullsized":"2024-02-18T11:48:47.305500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304586+0000","last_change":"2024-02-18T11:48:47.304586+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304586+0000","last_undegraded":"2024-02-18T11:48:47.304586+0000","last_fullsized":"2024-02-18T11:48:47.304586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305254+0000","last_change":"2024-02-18T11:48:47.305254+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305254+0000","last_undegraded":"2024-02-18T11:48:47.305254+0000","last_fullsized":"2024-02-18T11:48:47.305254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.300930+0000","last_change":"2024-02-18T11:48:47.300930+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.300930+0000","last_undegraded":"2024-02-18T11:48:47.300930+0000","last_fullsized":"2024-02-18T11:48:47.300930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.315231+0000","last_change":"2024-02-18T11:48:47.315173+0000","last_active":"2024-02-18T11:48:47.315231+0000","last_peered":"2024-02-18T11:48:47.315231+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.315173+0000","last_became_peered":"2024-02-18T11:48:47.315173+0000","last_unstale":"2024-02-18T11:48:47.315231+0000","last_undegraded":"2024-02-18T11:48:47.315231+0000","last_fullsized":"2024-02-18T11:48:47.315231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.310414+0000","last_change":"2024-02-18T11:48:47.310414+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.310414+0000","last_undegraded":"2024-02-18T11:48:47.310414+0000","last_fullsized":"2024-02-18T11:48:47.310414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310027+0000","last_change":"2024-02-18T11:48:47.309300+0000","last_active":"2024-02-18T11:48:47.310027+0000","last_peered":"2024-02-18T11:48:47.310027+0000","last_clean":"2024-02-18T11:48:47.310027+0000","last_became_active":"2024-02-18T11:48:47.309025+0000","last_became_peered":"2024-02-18T11:48:47.309025+0000","last_unstale":"2024-02-18T11:48:47.310027+0000","last_undegraded":"2024-02-18T11:48:47.310027+0000","last_fullsized":"2024-02-18T11:48:47.310027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306092+0000","last_change":"2024-02-18T11:48:47.306092+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306092+0000","last_undegraded":"2024-02-18T11:48:47.306092+0000","last_fullsized":"2024-02-18T11:48:47.306092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301304+0000","last_change":"2024-02-18T11:48:47.301304+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301304+0000","last_undegraded":"2024-02-18T11:48:47.301304+0000","last_fullsized":"2024-02-18T11:48:47.301304+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311897+0000","last_change":"2024-02-18T11:48:47.311897+0000","last_active":"2024-02-18T11:48:47.311897+0000","last_peered":"2024-02-18T11:48:47.311897+0000","last_clean":"2024-02-18T11:48:47.311897+0000","last_became_active":"2024-02-18T11:48:47.311571+0000","last_became_peered":"2024-02-18T11:48:47.311571+0000","last_unstale":"2024-02-18T11:48:47.311897+0000","last_undegraded":"2024-02-18T11:48:47.311897+0000","last_fullsized":"2024-02-18T11:48:47.311897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:46.613859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305634+0000","last_change":"2024-02-18T11:48:47.305634+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305634+0000","last_undegraded":"2024-02-18T11:48:47.305634+0000","last_fullsized":"2024-02-18T11:48:47.305634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305480+0000","last_change":"2024-02-18T11:48:47.305480+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305480+0000","last_undegraded":"2024-02-18T11:48:47.305480+0000","last_fullsized":"2024-02-18T11:48:47.305480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307947+0000","last_change":"2024-02-18T11:48:47.307947+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307947+0000","last_undegraded":"2024-02-18T11:48:47.307947+0000","last_fullsized":"2024-02-18T11:48:47.307947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301842+0000","last_change":"2024-02-18T11:48:46.297461+0000","last_active":"2024-02-18T11:48:47.301842+0000","last_peered":"2024-02-18T11:48:47.301842+0000","last_clean":"2024-02-18T11:48:47.301842+0000","last_became_active":"2024-02-18T11:48:46.297166+0000","last_became_peered":"2024-02-18T11:48:46.297166+0000","last_unstale":"2024-02-18T11:48:47.301842+0000","last_undegraded":"2024-02-18T11:48:47.301842+0000","last_fullsized":"2024-02-18T11:48:47.301842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306066+0000","last_change":"2024-02-18T11:48:47.306066+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306066+0000","last_undegraded":"2024-02-18T11:48:47.306066+0000","last_fullsized":"2024-02-18T11:48:47.306066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305368+0000","last_change":"2024-02-18T11:48:46.300302+0000","last_active":"2024-02-18T11:48:47.305368+0000","last_peered":"2024-02-18T11:48:47.305368+0000","last_clean":"2024-02-18T11:48:47.305368+0000","last_became_active":"2024-02-18T11:48:46.299909+0000","last_became_peered":"2024-02-18T11:48:46.299909+0000","last_unstale":"2024-02-18T11:48:47.305368+0000","last_undegraded":"2024-02-18T11:48:47.305368+0000","last_fullsized":"2024-02-18T11:48:47.305368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319795+0000","last_change":"2024-02-18T11:48:47.316579+0000","last_active":"2024-02-18T11:48:47.319795+0000","last_peered":"2024-02-18T11:48:47.319795+0000","last_clean":"2024-02-18T11:48:47.319795+0000","last_became_active":"2024-02-18T11:48:47.316195+0000","last_became_peered":"2024-02-18T11:48:47.316195+0000","last_unstale":"2024-02-18T11:48:47.319795+0000","last_undegraded":"2024-02-18T11:48:47.319795+0000","last_fullsized":"2024-02-18T11:48:47.319795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:18.861179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310566+0000","last_change":"2024-02-18T11:48:46.297756+0000","last_active":"2024-02-18T11:48:47.310566+0000","last_peered":"2024-02-18T11:48:47.310566+0000","last_clean":"2024-02-18T11:48:47.310566+0000","last_became_active":"2024-02-18T11:48:46.297485+0000","last_became_peered":"2024-02-18T11:48:46.297485+0000","last_unstale":"2024-02-18T11:48:47.310566+0000","last_undegraded":"2024-02-18T11:48:47.310566+0000","last_fullsized":"2024-02-18T11:48:47.310566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317072+0000","last_change":"2024-02-18T11:48:47.317072+0000","last_active":"2024-02-18T11:48:47.317072+0000","last_peered":"2024-02-18T11:48:47.317072+0000","last_clean":"2024-02-18T11:48:47.317072+0000","last_became_active":"2024-02-18T11:48:47.316360+0000","last_became_peered":"2024-02-18T11:48:47.316360+0000","last_unstale":"2024-02-18T11:48:47.317072+0000","last_undegraded":"2024-02-18T11:48:47.317072+0000","last_fullsized":"2024-02-18T11:48:47.317072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:08:26.674078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310420+0000","last_change":"2024-02-18T11:48:46.305539+0000","last_active":"2024-02-18T11:48:47.310420+0000","last_peered":"2024-02-18T11:48:47.310420+0000","last_clean":"2024-02-18T11:48:47.310420+0000","last_became_active":"2024-02-18T11:48:46.305252+0000","last_became_peered":"2024-02-18T11:48:46.305252+0000","last_unstale":"2024-02-18T11:48:47.310420+0000","last_undegraded":"2024-02-18T11:48:47.310420+0000","last_fullsized":"2024-02-18T11:48:47.310420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309454+0000","last_change":"2024-02-18T11:48:47.309454+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309454+0000","last_undegraded":"2024-02-18T11:48:47.309454+0000","last_fullsized":"2024-02-18T11:48:47.309454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305829+0000","last_change":"2024-02-18T11:48:46.308595+0000","last_active":"2024-02-18T11:48:47.305829+0000","last_peered":"2024-02-18T11:48:47.305829+0000","last_clean":"2024-02-18T11:48:47.305829+0000","last_became_active":"2024-02-18T11:48:46.308361+0000","last_became_peered":"2024-02-18T11:48:46.308361+0000","last_unstale":"2024-02-18T11:48:47.305829+0000","last_undegraded":"2024-02-18T11:48:47.305829+0000","last_fullsized":"2024-02-18T11:48:47.305829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305889+0000","last_change":"2024-02-18T11:48:47.305889+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305889+0000","last_undegraded":"2024-02-18T11:48:47.305889+0000","last_fullsized":"2024-02-18T11:48:47.305889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308401+0000","last_change":"2024-02-18T11:48:46.300103+0000","last_active":"2024-02-18T11:48:47.308401+0000","last_peered":"2024-02-18T11:48:47.308401+0000","last_clean":"2024-02-18T11:48:47.308401+0000","last_became_active":"2024-02-18T11:48:46.299808+0000","last_became_peered":"2024-02-18T11:48:46.299808+0000","last_unstale":"2024-02-18T11:48:47.308401+0000","last_undegraded":"2024-02-18T11:48:47.308401+0000","last_fullsized":"2024-02-18T11:48:47.308401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320639+0000","last_change":"2024-02-18T11:48:47.320639+0000","last_active":"2024-02-18T11:48:47.320639+0000","last_peered":"2024-02-18T11:48:47.320639+0000","last_clean":"2024-02-18T11:48:47.320639+0000","last_became_active":"2024-02-18T11:48:47.320385+0000","last_became_peered":"2024-02-18T11:48:47.320385+0000","last_unstale":"2024-02-18T11:48:47.320639+0000","last_undegraded":"2024-02-18T11:48:47.320639+0000","last_fullsized":"2024-02-18T11:48:47.320639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:57.144454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310023+0000","last_change":"2024-02-18T11:48:46.304701+0000","last_active":"2024-02-18T11:48:47.310023+0000","last_peered":"2024-02-18T11:48:47.310023+0000","last_clean":"2024-02-18T11:48:47.310023+0000","last_became_active":"2024-02-18T11:48:46.304450+0000","last_became_peered":"2024-02-18T11:48:46.304450+0000","last_unstale":"2024-02-18T11:48:47.310023+0000","last_undegraded":"2024-02-18T11:48:47.310023+0000","last_fullsized":"2024-02-18T11:48:47.310023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317646+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:47.317646+0000","last_became_active":"2024-02-18T11:48:47.317171+0000","last_became_peered":"2024-02-18T11:48:47.317171+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:14.550912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304637+0000","last_change":"2024-02-18T11:48:46.296450+0000","last_active":"2024-02-18T11:48:47.304637+0000","last_peered":"2024-02-18T11:48:47.304637+0000","last_clean":"2024-02-18T11:48:47.304637+0000","last_became_active":"2024-02-18T11:48:46.296159+0000","last_became_peered":"2024-02-18T11:48:46.296159+0000","last_unstale":"2024-02-18T11:48:47.304637+0000","last_undegraded":"2024-02-18T11:48:47.304637+0000","last_fullsized":"2024-02-18T11:48:47.304637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303802+0000","last_change":"2024-02-18T11:48:46.306411+0000","last_active":"2024-02-18T11:48:47.303802+0000","last_peered":"2024-02-18T11:48:47.303802+0000","last_clean":"2024-02-18T11:48:47.303802+0000","last_became_active":"2024-02-18T11:48:46.304718+0000","last_became_peered":"2024-02-18T11:48:46.304718+0000","last_unstale":"2024-02-18T11:48:47.303802+0000","last_undegraded":"2024-02-18T11:48:47.303802+0000","last_fullsized":"2024-02-18T11:48:47.303802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311877+0000","last_change":"2024-02-18T11:48:47.311877+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311877+0000","last_undegraded":"2024-02-18T11:48:47.311877+0000","last_fullsized":"2024-02-18T11:48:47.311877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308363+0000","last_change":"2024-02-18T11:48:46.307561+0000","last_active":"2024-02-18T11:48:47.308363+0000","last_peered":"2024-02-18T11:48:47.308363+0000","last_clean":"2024-02-18T11:48:47.308363+0000","last_became_active":"2024-02-18T11:48:46.307253+0000","last_became_peered":"2024-02-18T11:48:46.307253+0000","last_unstale":"2024-02-18T11:48:47.308363+0000","last_undegraded":"2024-02-18T11:48:47.308363+0000","last_fullsized":"2024-02-18T11:48:47.308363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.316738+0000","last_change":"2024-02-18T11:48:47.316738+0000","last_active":"2024-02-18T11:48:47.316738+0000","last_peered":"2024-02-18T11:48:47.316738+0000","last_clean":"2024-02-18T11:48:47.316738+0000","last_became_active":"2024-02-18T11:48:47.316488+0000","last_became_peered":"2024-02-18T11:48:47.316488+0000","last_unstale":"2024-02-18T11:48:47.316738+0000","last_undegraded":"2024-02-18T11:48:47.316738+0000","last_fullsized":"2024-02-18T11:48:47.316738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305690+0000","last_change":"2024-02-18T11:48:46.304153+0000","last_active":"2024-02-18T11:48:47.305690+0000","last_peered":"2024-02-18T11:48:47.305690+0000","last_clean":"2024-02-18T11:48:47.305690+0000","last_became_active":"2024-02-18T11:48:46.303687+0000","last_became_peered":"2024-02-18T11:48:46.303687+0000","last_unstale":"2024-02-18T11:48:47.305690+0000","last_undegraded":"2024-02-18T11:48:47.305690+0000","last_fullsized":"2024-02-18T11:48:47.305690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307009+0000","last_change":"2024-02-18T11:48:47.307009+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307009+0000","last_undegraded":"2024-02-18T11:48:47.307009+0000","last_fullsized":"2024-02-18T11:48:47.307009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304472+0000","last_change":"2024-02-18T11:48:46.301330+0000","last_active":"2024-02-18T11:48:47.304472+0000","last_peered":"2024-02-18T11:48:47.304472+0000","last_clean":"2024-02-18T11:48:47.304472+0000","last_became_active":"2024-02-18T11:48:46.301062+0000","last_became_peered":"2024-02-18T11:48:46.301062+0000","last_unstale":"2024-02-18T11:48:47.304472+0000","last_undegraded":"2024-02-18T11:48:47.304472+0000","last_fullsized":"2024-02-18T11:48:47.304472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309147+0000","last_change":"2024-02-18T11:48:47.309147+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309147+0000","last_undegraded":"2024-02-18T11:48:47.309147+0000","last_fullsized":"2024-02-18T11:48:47.309147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304790+0000","last_change":"2024-02-18T11:48:46.307286+0000","last_active":"2024-02-18T11:48:47.304790+0000","last_peered":"2024-02-18T11:48:47.304790+0000","last_clean":"2024-02-18T11:48:47.304790+0000","last_became_active":"2024-02-18T11:48:46.307002+0000","last_became_peered":"2024-02-18T11:48:46.307002+0000","last_unstale":"2024-02-18T11:48:47.304790+0000","last_undegraded":"2024-02-18T11:48:47.304790+0000","last_fullsized":"2024-02-18T11:48:47.304790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:21.137008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323364+0000","last_change":"2024-02-18T11:48:47.323364+0000","last_active":"2024-02-18T11:48:47.323364+0000","last_peered":"2024-02-18T11:48:47.323364+0000","last_clean":"2024-02-18T11:48:47.323364+0000","last_became_active":"2024-02-18T11:48:47.318734+0000","last_became_peered":"2024-02-18T11:48:47.318734+0000","last_unstale":"2024-02-18T11:48:47.323364+0000","last_undegraded":"2024-02-18T11:48:47.323364+0000","last_fullsized":"2024-02-18T11:48:47.323364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:29:29.908143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306130+0000","last_change":"2024-02-18T11:48:46.305434+0000","last_active":"2024-02-18T11:48:47.306130+0000","last_peered":"2024-02-18T11:48:47.306130+0000","last_clean":"2024-02-18T11:48:47.306130+0000","last_became_active":"2024-02-18T11:48:46.305145+0000","last_became_peered":"2024-02-18T11:48:46.305145+0000","last_unstale":"2024-02-18T11:48:47.306130+0000","last_undegraded":"2024-02-18T11:48:47.306130+0000","last_fullsized":"2024-02-18T11:48:47.306130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:07.775308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308255+0000","last_change":"2024-02-18T11:48:47.308255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308255+0000","last_undegraded":"2024-02-18T11:48:47.308255+0000","last_fullsized":"2024-02-18T11:48:47.308255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301908+0000","last_change":"2024-02-18T11:48:46.297664+0000","last_active":"2024-02-18T11:48:47.301908+0000","last_peered":"2024-02-18T11:48:47.301908+0000","last_clean":"2024-02-18T11:48:47.301908+0000","last_became_active":"2024-02-18T11:48:46.297202+0000","last_became_peered":"2024-02-18T11:48:46.297202+0000","last_unstale":"2024-02-18T11:48:47.301908+0000","last_undegraded":"2024-02-18T11:48:47.301908+0000","last_fullsized":"2024-02-18T11:48:47.301908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.318448+0000","last_change":"2024-02-18T11:48:47.318448+0000","last_active":"2024-02-18T11:48:47.318448+0000","last_peered":"2024-02-18T11:48:47.318448+0000","last_clean":"2024-02-18T11:48:47.318448+0000","last_became_active":"2024-02-18T11:48:47.318190+0000","last_became_peered":"2024-02-18T11:48:47.318190+0000","last_unstale":"2024-02-18T11:48:47.318448+0000","last_undegraded":"2024-02-18T11:48:47.318448+0000","last_fullsized":"2024-02-18T11:48:47.318448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:15:45.332480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306501+0000","last_change":"2024-02-18T11:48:46.300667+0000","last_active":"2024-02-18T11:48:47.306501+0000","last_peered":"2024-02-18T11:48:47.306501+0000","last_clean":"2024-02-18T11:48:47.306501+0000","last_became_active":"2024-02-18T11:48:46.300407+0000","last_became_peered":"2024-02-18T11:48:46.300407+0000","last_unstale":"2024-02-18T11:48:47.306501+0000","last_undegraded":"2024-02-18T11:48:47.306501+0000","last_fullsized":"2024-02-18T11:48:47.306501+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323994+0000","last_change":"2024-02-18T11:48:47.309612+0000","last_active":"2024-02-18T11:48:47.323994+0000","last_peered":"2024-02-18T11:48:47.323994+0000","last_clean":"2024-02-18T11:48:47.323994+0000","last_became_active":"2024-02-18T11:48:47.309382+0000","last_became_peered":"2024-02-18T11:48:47.309382+0000","last_unstale":"2024-02-18T11:48:47.323994+0000","last_undegraded":"2024-02-18T11:48:47.323994+0000","last_fullsized":"2024-02-18T11:48:47.323994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311657+0000","last_change":"2024-02-18T11:48:46.307646+0000","last_active":"2024-02-18T11:48:47.311657+0000","last_peered":"2024-02-18T11:48:47.311657+0000","last_clean":"2024-02-18T11:48:47.311657+0000","last_became_active":"2024-02-18T11:48:46.307345+0000","last_became_peered":"2024-02-18T11:48:46.307345+0000","last_unstale":"2024-02-18T11:48:47.311657+0000","last_undegraded":"2024-02-18T11:48:47.311657+0000","last_fullsized":"2024-02-18T11:48:47.311657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:50.554971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304896+0000","last_change":"2024-02-18T11:48:47.304896+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304896+0000","last_undegraded":"2024-02-18T11:48:47.304896+0000","last_fullsized":"2024-02-18T11:48:47.304896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306767+0000","last_change":"2024-02-18T11:48:46.306743+0000","last_active":"2024-02-18T11:48:47.306767+0000","last_peered":"2024-02-18T11:48:47.306767+0000","last_clean":"2024-02-18T11:48:47.306767+0000","last_became_active":"2024-02-18T11:48:46.306447+0000","last_became_peered":"2024-02-18T11:48:46.306447+0000","last_unstale":"2024-02-18T11:48:47.306767+0000","last_undegraded":"2024-02-18T11:48:47.306767+0000","last_fullsized":"2024-02-18T11:48:47.306767+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.321271+0000","last_change":"2024-02-18T11:48:47.321271+0000","last_active":"2024-02-18T11:48:47.321271+0000","last_peered":"2024-02-18T11:48:47.321271+0000","last_clean":"2024-02-18T11:48:47.321271+0000","last_became_active":"2024-02-18T11:48:47.321021+0000","last_became_peered":"2024-02-18T11:48:47.321021+0000","last_unstale":"2024-02-18T11:48:47.321271+0000","last_undegraded":"2024-02-18T11:48:47.321271+0000","last_fullsized":"2024-02-18T11:48:47.321271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301977+0000","last_change":"2024-02-18T11:48:46.296023+0000","last_active":"2024-02-18T11:48:47.301977+0000","last_peered":"2024-02-18T11:48:47.301977+0000","last_clean":"2024-02-18T11:48:47.301977+0000","last_became_active":"2024-02-18T11:48:46.295715+0000","last_became_peered":"2024-02-18T11:48:46.295715+0000","last_unstale":"2024-02-18T11:48:47.301977+0000","last_undegraded":"2024-02-18T11:48:47.301977+0000","last_fullsized":"2024-02-18T11:48:47.301977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310131+0000","last_change":"2024-02-18T11:48:47.310131+0000","last_active":"2024-02-18T11:48:47.310131+0000","last_peered":"2024-02-18T11:48:47.310131+0000","last_clean":"2024-02-18T11:48:47.310131+0000","last_became_active":"2024-02-18T11:48:47.309877+0000","last_became_peered":"2024-02-18T11:48:47.309877+0000","last_unstale":"2024-02-18T11:48:47.310131+0000","last_undegraded":"2024-02-18T11:48:47.310131+0000","last_fullsized":"2024-02-18T11:48:47.310131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305646+0000","last_change":"2024-02-18T11:48:46.297720+0000","last_active":"2024-02-18T11:48:47.305646+0000","last_peered":"2024-02-18T11:48:47.305646+0000","last_clean":"2024-02-18T11:48:47.305646+0000","last_became_active":"2024-02-18T11:48:46.297462+0000","last_became_peered":"2024-02-18T11:48:46.297462+0000","last_unstale":"2024-02-18T11:48:47.305646+0000","last_undegraded":"2024-02-18T11:48:47.305646+0000","last_fullsized":"2024-02-18T11:48:47.305646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306864+0000","last_change":"2024-02-18T11:48:47.306864+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306864+0000","last_undegraded":"2024-02-18T11:48:47.306864+0000","last_fullsized":"2024-02-18T11:48:47.306864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301800+0000","last_change":"2024-02-18T11:48:46.303136+0000","last_active":"2024-02-18T11:48:47.301800+0000","last_peered":"2024-02-18T11:48:47.301800+0000","last_clean":"2024-02-18T11:48:47.301800+0000","last_became_active":"2024-02-18T11:48:46.302861+0000","last_became_peered":"2024-02-18T11:48:46.302861+0000","last_unstale":"2024-02-18T11:48:47.301800+0000","last_undegraded":"2024-02-18T11:48:47.301800+0000","last_fullsized":"2024-02-18T11:48:47.301800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306944+0000","last_change":"2024-02-18T11:48:47.306944+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306944+0000","last_undegraded":"2024-02-18T11:48:47.306944+0000","last_fullsized":"2024-02-18T11:48:47.306944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.300950+0000","last_change":"2024-02-18T11:48:46.297761+0000","last_active":"2024-02-18T11:48:47.300950+0000","last_peered":"2024-02-18T11:48:47.300950+0000","last_clean":"2024-02-18T11:48:47.300950+0000","last_became_active":"2024-02-18T11:48:46.297500+0000","last_became_peered":"2024-02-18T11:48:46.297500+0000","last_unstale":"2024-02-18T11:48:47.300950+0000","last_undegraded":"2024-02-18T11:48:47.300950+0000","last_fullsized":"2024-02-18T11:48:47.300950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:33:10.032538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310081+0000","last_change":"2024-02-18T11:48:47.310081+0000","last_active":"2024-02-18T11:48:47.310081+0000","last_peered":"2024-02-18T11:48:47.310081+0000","last_clean":"2024-02-18T11:48:47.310081+0000","last_became_active":"2024-02-18T11:48:47.309813+0000","last_became_peered":"2024-02-18T11:48:47.309813+0000","last_unstale":"2024-02-18T11:48:47.310081+0000","last_undegraded":"2024-02-18T11:48:47.310081+0000","last_fullsized":"2024-02-18T11:48:47.310081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306548+0000","last_change":"2024-02-18T11:48:46.298569+0000","last_active":"2024-02-18T11:48:47.306548+0000","last_peered":"2024-02-18T11:48:47.306548+0000","last_clean":"2024-02-18T11:48:47.306548+0000","last_became_active":"2024-02-18T11:48:46.297865+0000","last_became_peered":"2024-02-18T11:48:46.297865+0000","last_unstale":"2024-02-18T11:48:47.306548+0000","last_undegraded":"2024-02-18T11:48:47.306548+0000","last_fullsized":"2024-02-18T11:48:47.306548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:04.067605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.312852+0000","last_change":"2024-02-18T11:48:47.312852+0000","last_active":"2024-02-18T11:48:47.312852+0000","last_peered":"2024-02-18T11:48:47.312852+0000","last_clean":"2024-02-18T11:48:47.312852+0000","last_became_active":"2024-02-18T11:48:47.312584+0000","last_became_peered":"2024-02-18T11:48:47.312584+0000","last_unstale":"2024-02-18T11:48:47.312852+0000","last_undegraded":"2024-02-18T11:48:47.312852+0000","last_fullsized":"2024-02-18T11:48:47.312852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:51.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301208+0000","last_change":"2024-02-18T11:48:46.302429+0000","last_active":"2024-02-18T11:48:47.301208+0000","last_peered":"2024-02-18T11:48:47.301208+0000","last_clean":"2024-02-18T11:48:47.301208+0000","last_became_active":"2024-02-18T11:48:46.302138+0000","last_became_peered":"2024-02-18T11:48:46.302138+0000","last_unstale":"2024-02-18T11:48:47.301208+0000","last_undegraded":"2024-02-18T11:48:47.301208+0000","last_fullsized":"2024-02-18T11:48:47.301208+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:54.091351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305364+0000","last_change":"2024-02-18T11:48:47.305364+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305364+0000","last_undegraded":"2024-02-18T11:48:47.305364+0000","last_fullsized":"2024-02-18T11:48:47.305364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311359+0000","last_change":"2024-02-18T11:48:46.308271+0000","last_active":"2024-02-18T11:48:47.311359+0000","last_peered":"2024-02-18T11:48:47.311359+0000","last_clean":"2024-02-18T11:48:47.311359+0000","last_became_active":"2024-02-18T11:48:46.307981+0000","last_became_peered":"2024-02-18T11:48:46.307981+0000","last_unstale":"2024-02-18T11:48:47.311359+0000","last_undegraded":"2024-02-18T11:48:47.311359+0000","last_fullsized":"2024-02-18T11:48:47.311359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:32.990774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311673+0000","last_change":"2024-02-18T11:48:47.311673+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311673+0000","last_undegraded":"2024-02-18T11:48:47.311673+0000","last_fullsized":"2024-02-18T11:48:47.311673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301714+0000","last_change":"2024-02-18T11:48:46.305551+0000","last_active":"2024-02-18T11:48:47.301714+0000","last_peered":"2024-02-18T11:48:47.301714+0000","last_clean":"2024-02-18T11:48:47.301714+0000","last_became_active":"2024-02-18T11:48:46.305246+0000","last_became_peered":"2024-02-18T11:48:46.305246+0000","last_unstale":"2024-02-18T11:48:47.301714+0000","last_undegraded":"2024-02-18T11:48:47.301714+0000","last_fullsized":"2024-02-18T11:48:47.301714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319083+0000","last_change":"2024-02-18T11:48:47.319083+0000","last_active":"2024-02-18T11:48:47.319083+0000","last_peered":"2024-02-18T11:48:47.319083+0000","last_clean":"2024-02-18T11:48:47.319083+0000","last_became_active":"2024-02-18T11:48:47.318762+0000","last_became_peered":"2024-02-18T11:48:47.318762+0000","last_unstale":"2024-02-18T11:48:47.319083+0000","last_undegraded":"2024-02-18T11:48:47.319083+0000","last_fullsized":"2024-02-18T11:48:47.319083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:55:35.888377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305354+0000","last_change":"2024-02-18T11:48:46.305494+0000","last_active":"2024-02-18T11:48:47.305354+0000","last_peered":"2024-02-18T11:48:47.305354+0000","last_clean":"2024-02-18T11:48:47.305354+0000","last_became_active":"2024-02-18T11:48:46.305222+0000","last_became_peered":"2024-02-18T11:48:46.305222+0000","last_unstale":"2024-02-18T11:48:47.305354+0000","last_undegraded":"2024-02-18T11:48:47.305354+0000","last_fullsized":"2024-02-18T11:48:47.305354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308833+0000","last_change":"2024-02-18T11:48:47.308833+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308833+0000","last_undegraded":"2024-02-18T11:48:47.308833+0000","last_fullsized":"2024-02-18T11:48:47.308833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308558+0000","last_change":"2024-02-18T11:48:46.303722+0000","last_active":"2024-02-18T11:48:47.308558+0000","last_peered":"2024-02-18T11:48:47.308558+0000","last_clean":"2024-02-18T11:48:47.308558+0000","last_became_active":"2024-02-18T11:48:46.303460+0000","last_became_peered":"2024-02-18T11:48:46.303460+0000","last_unstale":"2024-02-18T11:48:47.308558+0000","last_undegraded":"2024-02-18T11:48:47.308558+0000","last_fullsized":"2024-02-18T11:48:47.308558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309759+0000","last_change":"2024-02-18T11:48:47.309759+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309759+0000","last_undegraded":"2024-02-18T11:48:47.309759+0000","last_fullsized":"2024-02-18T11:48:47.309759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.302363+0000","last_change":"2024-02-18T11:48:46.299505+0000","last_active":"2024-02-18T11:48:47.302363+0000","last_peered":"2024-02-18T11:48:47.302363+0000","last_clean":"2024-02-18T11:48:47.302363+0000","last_became_active":"2024-02-18T11:48:46.299230+0000","last_became_peered":"2024-02-18T11:48:46.299230+0000","last_unstale":"2024-02-18T11:48:47.302363+0000","last_undegraded":"2024-02-18T11:48:47.302363+0000","last_fullsized":"2024-02-18T11:48:47.302363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320820+0000","last_change":"2024-02-18T11:48:47.315114+0000","last_active":"2024-02-18T11:48:47.320820+0000","last_peered":"2024-02-18T11:48:47.320820+0000","last_clean":"2024-02-18T11:48:47.320820+0000","last_became_active":"2024-02-18T11:48:47.313064+0000","last_became_peered":"2024-02-18T11:48:47.313064+0000","last_unstale":"2024-02-18T11:48:47.320820+0000","last_undegraded":"2024-02-18T11:48:47.320820+0000","last_fullsized":"2024-02-18T11:48:47.320820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:22.773038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307959+0000","last_change":"2024-02-18T11:48:46.306424+0000","last_active":"2024-02-18T11:48:47.307959+0000","last_peered":"2024-02-18T11:48:47.307959+0000","last_clean":"2024-02-18T11:48:47.307959+0000","last_became_active":"2024-02-18T11:48:46.305938+0000","last_became_peered":"2024-02-18T11:48:46.305938+0000","last_unstale":"2024-02-18T11:48:47.307959+0000","last_undegraded":"2024-02-18T11:48:47.307959+0000","last_fullsized":"2024-02-18T11:48:47.307959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:48.967965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.314001+0000","last_change":"2024-02-18T11:48:47.314001+0000","last_active":"2024-02-18T11:48:47.314001+0000","last_peered":"2024-02-18T11:48:47.314001+0000","last_clean":"2024-02-18T11:48:47.314001+0000","last_became_active":"2024-02-18T11:48:47.313680+0000","last_became_peered":"2024-02-18T11:48:47.313680+0000","last_unstale":"2024-02-18T11:48:47.314001+0000","last_undegraded":"2024-02-18T11:48:47.314001+0000","last_fullsized":"2024-02-18T11:48:47.314001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303228+0000","last_change":"2024-02-18T11:48:46.306107+0000","last_active":"2024-02-18T11:48:47.303228+0000","last_peered":"2024-02-18T11:48:47.303228+0000","last_clean":"2024-02-18T11:48:47.303228+0000","last_became_active":"2024-02-18T11:48:46.305749+0000","last_became_peered":"2024-02-18T11:48:46.305749+0000","last_unstale":"2024-02-18T11:48:47.303228+0000","last_undegraded":"2024-02-18T11:48:47.303228+0000","last_fullsized":"2024-02-18T11:48:47.303228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311076+0000","last_change":"2024-02-18T11:48:47.311076+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311076+0000","last_undegraded":"2024-02-18T11:48:47.311076+0000","last_fullsized":"2024-02-18T11:48:47.311076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307802+0000","last_change":"2024-02-18T11:48:46.303447+0000","last_active":"2024-02-18T11:48:47.307802+0000","last_peered":"2024-02-18T11:48:47.307802+0000","last_clean":"2024-02-18T11:48:47.307802+0000","last_became_active":"2024-02-18T11:48:46.303126+0000","last_became_peered":"2024-02-18T11:48:46.303126+0000","last_unstale":"2024-02-18T11:48:47.307802+0000","last_undegraded":"2024-02-18T11:48:47.307802+0000","last_fullsized":"2024-02-18T11:48:47.307802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:44.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307052+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:48:47.307052+0000","last_peered":"2024-02-18T11:48:47.307052+0000","last_clean":"2024-02-18T11:48:47.307052+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:48:47.307052+0000","last_undegraded":"2024-02-18T11:48:47.307052+0000","last_fullsized":"2024-02-18T11:48:47.307052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21,"ondisk_log_size":21,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:50.598 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:50.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.743+0000 7f7da0702640 1 Processor -- start 2024-02-18T11:48:50.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.743+0000 7f7da0702640 1 -- start start 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7da0702640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815a860 0x7f7d980cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7da0702640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d980cd1f0 0x7f7d9806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7da0702640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d9815ac40 0x7f7d9806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7da0702640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980b02e0 con 0x7f7d9815ac40 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7da0702640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980af6d0 con 0x7f7d980cd1f0 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7da0702640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980af9b0 con 0x7f7d9815a860 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d9815ac40 0x7f7d9806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d9815ac40 0x7f7d9806e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41968/0 (socket says 172.21.15.116:41968) 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 -- 172.21.15.116:0/2199499369 learned_addr learned my addr 172.21.15.116:0/2199499369 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 -- 172.21.15.116:0/2199499369 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815a860 msgr2=0x7f7d980cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 --2- 172.21.15.116:0/2199499369 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815a860 0x7f7d980cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9dc9d640 1 --2- 172.21.15.116:0/2199499369 >> v2:172.21.15.143:3300/0 conn(0x7f7d980cd1f0 0x7f7d9806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 -- 172.21.15.116:0/2199499369 >> v2:172.21.15.143:3300/0 conn(0x7f7d980cd1f0 msgr2=0x7f7d9806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 --2- 172.21.15.116:0/2199499369 >> v2:172.21.15.143:3300/0 conn(0x7f7d980cd1f0 0x7f7d9806e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.744+0000 7f7d9ec9f640 1 -- 172.21.15.116:0/2199499369 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d980afc90 con 0x7f7d9815ac40 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.745+0000 7f7d9ec9f640 1 --2- 172.21.15.116:0/2199499369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815ac40 0x7f7d9806e850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806ef50 tx=0x7f7d9002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91beda88cb01a7a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.745+0000 7f7d9d49c640 1 -- 172.21.15.116:0/2199499369 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9002ca70 con 0x7f7d9815ac40 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.745+0000 7f7d9d49c640 1 -- 172.21.15.116:0/2199499369 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9002cbd0 con 0x7f7d9815ac40 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.745+0000 7f7d9d49c640 1 -- 172.21.15.116:0/2199499369 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d900343f0 con 0x7f7d9815ac40 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.745+0000 7f7da0702640 1 -- 172.21.15.116:0/2199499369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815ac40 msgr2=0x7f7d9806e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.745+0000 7f7da0702640 1 --2- 172.21.15.116:0/2199499369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815ac40 0x7f7d9806e850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806ef50 tx=0x7f7d9002bed0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 -- 172.21.15.116:0/2199499369 shutdown_connections 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 --2- 172.21.15.116:0/2199499369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815ac40 0x7f7d9806e850 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 --2- 172.21.15.116:0/2199499369 >> v2:172.21.15.143:3300/0 conn(0x7f7d980cd1f0 0x7f7d9806e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 --2- 172.21.15.116:0/2199499369 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815a860 0x7f7d980cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.740 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 -- 172.21.15.116:0/2199499369 >> 172.21.15.116:0/2199499369 conn(0x7f7d98066f20 msgr2=0x7f7d9806ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:50.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 -- 172.21.15.116:0/2199499369 shutdown_connections 2024-02-18T11:48:50.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.746+0000 7f7da0702640 1 -- 172.21.15.116:0/2199499369 wait complete. 2024-02-18T11:48:50.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 Processor -- start 2024-02-18T11:48:50.743 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 -- start start 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d980cd1f0 0x7f7d9814e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 0x7f7d981521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815ac40 0x7f7d9814cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9814df50 con 0x7f7d9815a860 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d981554d0 con 0x7f7d980cd1f0 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.747+0000 7f7da0702640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9814d5b0 con 0x7f7d9815ac40 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 0x7f7d981521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 0x7f7d981521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:41982/0 (socket says 172.21.15.116:41982) 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 -- 172.21.15.116:0/3967229826 learned_addr learned my addr 172.21.15.116:0/3967229826 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9e49e640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d980cd1f0 0x7f7d9814e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9ec9f640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815ac40 0x7f7d9814cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 -- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815ac40 msgr2=0x7f7d9814cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815ac40 0x7f7d9814cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 -- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d980cd1f0 msgr2=0x7f7d9814e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d980cd1f0 0x7f7d9814e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 -- 172.21.15.116:0/3967229826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009b30 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9e49e640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d980cd1f0 0x7f7d9814e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d9dc9d640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 0x7f7d981521f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c7f0 tx=0x7f7d8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88004220 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d880043a0 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88007860 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9814d890 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.748+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d98154dd0 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.749+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9814e090 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.750+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f7d88007bd0 con 0x7f7d9815a860 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.750+0000 7f7d8effd640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d7c06bca0 0x7f7d7c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.750+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7d8808a890 con 0x7f7d9815a860 2024-02-18T11:48:50.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.750+0000 7f7d9e49e640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d7c06bca0 0x7f7d7c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:50.746 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.750+0000 7f7d9e49e640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d7c06bca0 0x7f7d7c06e160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.754+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d9814e090 con 0x7f7d9815a860 2024-02-18T11:48:50.916 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.922+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d9814e090 con 0x7f7d7c06bca0 2024-02-18T11:48:50.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.928+0000 7f7d8effd640 1 -- 172.21.15.116:0/3967229826 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248905 (secure 0 0 0) 0x7f7d9814e090 con 0x7f7d7c06bca0 2024-02-18T11:48:50.924 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:50.925 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d7c06bca0 msgr2=0x7f7d7c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d7c06bca0 0x7f7d7c06e160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d94006fd0 tx=0x7f7d94008040 comp rx=0 tx=0).stop 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 msgr2=0x7f7d981521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 0x7f7d981521f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800c7f0 tx=0x7f7d8800ccb0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 shutdown_connections 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f7d7c06bca0 0x7f7d7c06e160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7d9815ac40 0x7f7d9814cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d9815a860 0x7f7d981521f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 --2- 172.21.15.116:0/3967229826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d980cd1f0 0x7f7d9814e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 >> 172.21.15.116:0/3967229826 conn(0x7f7d98066f20 msgr2=0x7f7d98159240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 shutdown_connections 2024-02-18T11:48:50.927 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:50.931+0000 7f7da0702640 1 -- 172.21.15.116:0/3967229826 wait complete. 2024-02-18T11:48:50.934 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-02-18T11:48:50.306500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591656,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4509696,"data_stored":2598551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.024513"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319567+0000","last_change":"2024-02-18T11:48:47.319567+0000","last_active":"2024-02-18T11:48:47.319567+0000","last_peered":"2024-02-18T11:48:47.319567+0000","last_clean":"2024-02-18T11:48:47.319567+0000","last_became_active":"2024-02-18T11:48:47.318466+0000","last_became_peered":"2024-02-18T11:48:47.318466+0000","last_unstale":"2024-02-18T11:48:47.319567+0000","last_undegraded":"2024-02-18T11:48:47.319567+0000","last_fullsized":"2024-02-18T11:48:47.319567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:35:02.679276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317654+0000","last_change":"2024-02-18T11:48:47.317654+0000","last_active":"2024-02-18T11:48:47.317654+0000","last_peered":"2024-02-18T11:48:47.317654+0000","last_clean":"2024-02-18T11:48:47.317654+0000","last_became_active":"2024-02-18T11:48:47.316735+0000","last_became_peered":"2024-02-18T11:48:47.316735+0000","last_unstale":"2024-02-18T11:48:47.317654+0000","last_undegraded":"2024-02-18T11:48:47.317654+0000","last_fullsized":"2024-02-18T11:48:47.317654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305050+0000","last_change":"2024-02-18T11:48:47.305050+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305050+0000","last_undegraded":"2024-02-18T11:48:47.305050+0000","last_fullsized":"2024-02-18T11:48:47.305050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.302121+0000","last_change":"2024-02-18T11:48:47.302121+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.302121+0000","last_undegraded":"2024-02-18T11:48:47.302121+0000","last_fullsized":"2024-02-18T11:48:47.302121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301388+0000","last_change":"2024-02-18T11:48:47.301388+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301388+0000","last_undegraded":"2024-02-18T11:48:47.301388+0000","last_fullsized":"2024-02-18T11:48:47.301388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":4,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.313339+0000","last_change":"2024-02-18T11:48:47.305848+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.313339+0000","last_undegraded":"2024-02-18T11:48:47.313339+0000","last_fullsized":"2024-02-18T11:48:47.313339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304835+0000","last_change":"2024-02-18T11:48:47.304835+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304835+0000","last_undegraded":"2024-02-18T11:48:47.304835+0000","last_fullsized":"2024-02-18T11:48:47.304835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319168+0000","last_change":"2024-02-18T11:48:47.319168+0000","last_active":"2024-02-18T11:48:47.319168+0000","last_peered":"2024-02-18T11:48:47.319168+0000","last_clean":"2024-02-18T11:48:47.319168+0000","last_became_active":"2024-02-18T11:48:47.318918+0000","last_became_peered":"2024-02-18T11:48:47.318918+0000","last_unstale":"2024-02-18T11:48:47.319168+0000","last_undegraded":"2024-02-18T11:48:47.319168+0000","last_fullsized":"2024-02-18T11:48:47.319168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:03:08.526820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306155+0000","last_change":"2024-02-18T11:48:47.306155+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306155+0000","last_undegraded":"2024-02-18T11:48:47.306155+0000","last_fullsized":"2024-02-18T11:48:47.306155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317587+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.317587+0000","last_became_peered":"2024-02-18T11:48:47.317587+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305372+0000","last_change":"2024-02-18T11:48:47.305372+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305372+0000","last_undegraded":"2024-02-18T11:48:47.305372+0000","last_fullsized":"2024-02-18T11:48:47.305372+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305270+0000","last_change":"2024-02-18T11:48:47.305270+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305270+0000","last_undegraded":"2024-02-18T11:48:47.305270+0000","last_fullsized":"2024-02-18T11:48:47.305270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305144+0000","last_change":"2024-02-18T11:48:47.305144+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305144+0000","last_undegraded":"2024-02-18T11:48:47.305144+0000","last_fullsized":"2024-02-18T11:48:47.305144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320799+0000","last_change":"2024-02-18T11:48:47.314842+0000","last_active":"2024-02-18T11:48:47.320799+0000","last_peered":"2024-02-18T11:48:47.320799+0000","last_clean":"2024-02-18T11:48:47.320799+0000","last_became_active":"2024-02-18T11:48:47.312762+0000","last_became_peered":"2024-02-18T11:48:47.312762+0000","last_unstale":"2024-02-18T11:48:47.320799+0000","last_undegraded":"2024-02-18T11:48:47.320799+0000","last_fullsized":"2024-02-18T11:48:47.320799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:08.779058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304457+0000","last_change":"2024-02-18T11:48:47.304457+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304457+0000","last_undegraded":"2024-02-18T11:48:47.304457+0000","last_fullsized":"2024-02-18T11:48:47.304457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305255+0000","last_change":"2024-02-18T11:48:47.305255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305255+0000","last_undegraded":"2024-02-18T11:48:47.305255+0000","last_fullsized":"2024-02-18T11:48:47.305255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308258+0000","last_change":"2024-02-18T11:48:47.308258+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308258+0000","last_undegraded":"2024-02-18T11:48:47.308258+0000","last_fullsized":"2024-02-18T11:48:47.308258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301260+0000","last_change":"2024-02-18T11:48:47.301260+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301260+0000","last_undegraded":"2024-02-18T11:48:47.301260+0000","last_fullsized":"2024-02-18T11:48:47.301260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301207+0000","last_change":"2024-02-18T11:48:47.301207+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301207+0000","last_undegraded":"2024-02-18T11:48:47.301207+0000","last_fullsized":"2024-02-18T11:48:47.301207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305500+0000","last_change":"2024-02-18T11:48:47.305500+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305500+0000","last_undegraded":"2024-02-18T11:48:47.305500+0000","last_fullsized":"2024-02-18T11:48:47.305500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304586+0000","last_change":"2024-02-18T11:48:47.304586+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304586+0000","last_undegraded":"2024-02-18T11:48:47.304586+0000","last_fullsized":"2024-02-18T11:48:47.304586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305254+0000","last_change":"2024-02-18T11:48:47.305254+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305254+0000","last_undegraded":"2024-02-18T11:48:47.305254+0000","last_fullsized":"2024-02-18T11:48:47.305254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.300930+0000","last_change":"2024-02-18T11:48:47.300930+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.300930+0000","last_undegraded":"2024-02-18T11:48:47.300930+0000","last_fullsized":"2024-02-18T11:48:47.300930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.315231+0000","last_change":"2024-02-18T11:48:47.315173+0000","last_active":"2024-02-18T11:48:47.315231+0000","last_peered":"2024-02-18T11:48:47.315231+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.315173+0000","last_became_peered":"2024-02-18T11:48:47.315173+0000","last_unstale":"2024-02-18T11:48:47.315231+0000","last_undegraded":"2024-02-18T11:48:47.315231+0000","last_fullsized":"2024-02-18T11:48:47.315231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.310414+0000","last_change":"2024-02-18T11:48:47.310414+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.310414+0000","last_undegraded":"2024-02-18T11:48:47.310414+0000","last_fullsized":"2024-02-18T11:48:47.310414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310027+0000","last_change":"2024-02-18T11:48:47.309300+0000","last_active":"2024-02-18T11:48:47.310027+0000","last_peered":"2024-02-18T11:48:47.310027+0000","last_clean":"2024-02-18T11:48:47.310027+0000","last_became_active":"2024-02-18T11:48:47.309025+0000","last_became_peered":"2024-02-18T11:48:47.309025+0000","last_unstale":"2024-02-18T11:48:47.310027+0000","last_undegraded":"2024-02-18T11:48:47.310027+0000","last_fullsized":"2024-02-18T11:48:47.310027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306092+0000","last_change":"2024-02-18T11:48:47.306092+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306092+0000","last_undegraded":"2024-02-18T11:48:47.306092+0000","last_fullsized":"2024-02-18T11:48:47.306092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301304+0000","last_change":"2024-02-18T11:48:47.301304+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301304+0000","last_undegraded":"2024-02-18T11:48:47.301304+0000","last_fullsized":"2024-02-18T11:48:47.301304+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311897+0000","last_change":"2024-02-18T11:48:47.311897+0000","last_active":"2024-02-18T11:48:47.311897+0000","last_peered":"2024-02-18T11:48:47.311897+0000","last_clean":"2024-02-18T11:48:47.311897+0000","last_became_active":"2024-02-18T11:48:47.311571+0000","last_became_peered":"2024-02-18T11:48:47.311571+0000","last_unstale":"2024-02-18T11:48:47.311897+0000","last_undegraded":"2024-02-18T11:48:47.311897+0000","last_fullsized":"2024-02-18T11:48:47.311897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:46.613859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305634+0000","last_change":"2024-02-18T11:48:47.305634+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305634+0000","last_undegraded":"2024-02-18T11:48:47.305634+0000","last_fullsized":"2024-02-18T11:48:47.305634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305480+0000","last_change":"2024-02-18T11:48:47.305480+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305480+0000","last_undegraded":"2024-02-18T11:48:47.305480+0000","last_fullsized":"2024-02-18T11:48:47.305480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307947+0000","last_change":"2024-02-18T11:48:47.307947+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307947+0000","last_undegraded":"2024-02-18T11:48:47.307947+0000","last_fullsized":"2024-02-18T11:48:47.307947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301842+0000","last_change":"2024-02-18T11:48:46.297461+0000","last_active":"2024-02-18T11:48:47.301842+0000","last_peered":"2024-02-18T11:48:47.301842+0000","last_clean":"2024-02-18T11:48:47.301842+0000","last_became_active":"2024-02-18T11:48:46.297166+0000","last_became_peered":"2024-02-18T11:48:46.297166+0000","last_unstale":"2024-02-18T11:48:47.301842+0000","last_undegraded":"2024-02-18T11:48:47.301842+0000","last_fullsized":"2024-02-18T11:48:47.301842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306066+0000","last_change":"2024-02-18T11:48:47.306066+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306066+0000","last_undegraded":"2024-02-18T11:48:47.306066+0000","last_fullsized":"2024-02-18T11:48:47.306066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305368+0000","last_change":"2024-02-18T11:48:46.300302+0000","last_active":"2024-02-18T11:48:47.305368+0000","last_peered":"2024-02-18T11:48:47.305368+0000","last_clean":"2024-02-18T11:48:47.305368+0000","last_became_active":"2024-02-18T11:48:46.299909+0000","last_became_peered":"2024-02-18T11:48:46.299909+0000","last_unstale":"2024-02-18T11:48:47.305368+0000","last_undegraded":"2024-02-18T11:48:47.305368+0000","last_fullsized":"2024-02-18T11:48:47.305368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319795+0000","last_change":"2024-02-18T11:48:47.316579+0000","last_active":"2024-02-18T11:48:47.319795+0000","last_peered":"2024-02-18T11:48:47.319795+0000","last_clean":"2024-02-18T11:48:47.319795+0000","last_became_active":"2024-02-18T11:48:47.316195+0000","last_became_peered":"2024-02-18T11:48:47.316195+0000","last_unstale":"2024-02-18T11:48:47.319795+0000","last_undegraded":"2024-02-18T11:48:47.319795+0000","last_fullsized":"2024-02-18T11:48:47.319795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:18.861179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310566+0000","last_change":"2024-02-18T11:48:46.297756+0000","last_active":"2024-02-18T11:48:47.310566+0000","last_peered":"2024-02-18T11:48:47.310566+0000","last_clean":"2024-02-18T11:48:47.310566+0000","last_became_active":"2024-02-18T11:48:46.297485+0000","last_became_peered":"2024-02-18T11:48:46.297485+0000","last_unstale":"2024-02-18T11:48:47.310566+0000","last_undegraded":"2024-02-18T11:48:47.310566+0000","last_fullsized":"2024-02-18T11:48:47.310566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317072+0000","last_change":"2024-02-18T11:48:47.317072+0000","last_active":"2024-02-18T11:48:47.317072+0000","last_peered":"2024-02-18T11:48:47.317072+0000","last_clean":"2024-02-18T11:48:47.317072+0000","last_became_active":"2024-02-18T11:48:47.316360+0000","last_became_peered":"2024-02-18T11:48:47.316360+0000","last_unstale":"2024-02-18T11:48:47.317072+0000","last_undegraded":"2024-02-18T11:48:47.317072+0000","last_fullsized":"2024-02-18T11:48:47.317072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:08:26.674078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310420+0000","last_change":"2024-02-18T11:48:46.305539+0000","last_active":"2024-02-18T11:48:47.310420+0000","last_peered":"2024-02-18T11:48:47.310420+0000","last_clean":"2024-02-18T11:48:47.310420+0000","last_became_active":"2024-02-18T11:48:46.305252+0000","last_became_peered":"2024-02-18T11:48:46.305252+0000","last_unstale":"2024-02-18T11:48:47.310420+0000","last_undegraded":"2024-02-18T11:48:47.310420+0000","last_fullsized":"2024-02-18T11:48:47.310420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309454+0000","last_change":"2024-02-18T11:48:47.309454+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309454+0000","last_undegraded":"2024-02-18T11:48:47.309454+0000","last_fullsized":"2024-02-18T11:48:47.309454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305829+0000","last_change":"2024-02-18T11:48:46.308595+0000","last_active":"2024-02-18T11:48:47.305829+0000","last_peered":"2024-02-18T11:48:47.305829+0000","last_clean":"2024-02-18T11:48:47.305829+0000","last_became_active":"2024-02-18T11:48:46.308361+0000","last_became_peered":"2024-02-18T11:48:46.308361+0000","last_unstale":"2024-02-18T11:48:47.305829+0000","last_undegraded":"2024-02-18T11:48:47.305829+0000","last_fullsized":"2024-02-18T11:48:47.305829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305889+0000","last_change":"2024-02-18T11:48:47.305889+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305889+0000","last_undegraded":"2024-02-18T11:48:47.305889+0000","last_fullsized":"2024-02-18T11:48:47.305889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308401+0000","last_change":"2024-02-18T11:48:46.300103+0000","last_active":"2024-02-18T11:48:47.308401+0000","last_peered":"2024-02-18T11:48:47.308401+0000","last_clean":"2024-02-18T11:48:47.308401+0000","last_became_active":"2024-02-18T11:48:46.299808+0000","last_became_peered":"2024-02-18T11:48:46.299808+0000","last_unstale":"2024-02-18T11:48:47.308401+0000","last_undegraded":"2024-02-18T11:48:47.308401+0000","last_fullsized":"2024-02-18T11:48:47.308401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320639+0000","last_change":"2024-02-18T11:48:47.320639+0000","last_active":"2024-02-18T11:48:47.320639+0000","last_peered":"2024-02-18T11:48:47.320639+0000","last_clean":"2024-02-18T11:48:47.320639+0000","last_became_active":"2024-02-18T11:48:47.320385+0000","last_became_peered":"2024-02-18T11:48:47.320385+0000","last_unstale":"2024-02-18T11:48:47.320639+0000","last_undegraded":"2024-02-18T11:48:47.320639+0000","last_fullsized":"2024-02-18T11:48:47.320639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:57.144454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310023+0000","last_change":"2024-02-18T11:48:46.304701+0000","last_active":"2024-02-18T11:48:47.310023+0000","last_peered":"2024-02-18T11:48:47.310023+0000","last_clean":"2024-02-18T11:48:47.310023+0000","last_became_active":"2024-02-18T11:48:46.304450+0000","last_became_peered":"2024-02-18T11:48:46.304450+0000","last_unstale":"2024-02-18T11:48:47.310023+0000","last_undegraded":"2024-02-18T11:48:47.310023+0000","last_fullsized":"2024-02-18T11:48:47.310023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317646+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:47.317646+0000","last_became_active":"2024-02-18T11:48:47.317171+0000","last_became_peered":"2024-02-18T11:48:47.317171+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:14.550912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304637+0000","last_change":"2024-02-18T11:48:46.296450+0000","last_active":"2024-02-18T11:48:47.304637+0000","last_peered":"2024-02-18T11:48:47.304637+0000","last_clean":"2024-02-18T11:48:47.304637+0000","last_became_active":"2024-02-18T11:48:46.296159+0000","last_became_peered":"2024-02-18T11:48:46.296159+0000","last_unstale":"2024-02-18T11:48:47.304637+0000","last_undegraded":"2024-02-18T11:48:47.304637+0000","last_fullsized":"2024-02-18T11:48:47.304637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303802+0000","last_change":"2024-02-18T11:48:46.306411+0000","last_active":"2024-02-18T11:48:47.303802+0000","last_peered":"2024-02-18T11:48:47.303802+0000","last_clean":"2024-02-18T11:48:47.303802+0000","last_became_active":"2024-02-18T11:48:46.304718+0000","last_became_peered":"2024-02-18T11:48:46.304718+0000","last_unstale":"2024-02-18T11:48:47.303802+0000","last_undegraded":"2024-02-18T11:48:47.303802+0000","last_fullsized":"2024-02-18T11:48:47.303802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311877+0000","last_change":"2024-02-18T11:48:47.311877+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311877+0000","last_undegraded":"2024-02-18T11:48:47.311877+0000","last_fullsized":"2024-02-18T11:48:47.311877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308363+0000","last_change":"2024-02-18T11:48:46.307561+0000","last_active":"2024-02-18T11:48:47.308363+0000","last_peered":"2024-02-18T11:48:47.308363+0000","last_clean":"2024-02-18T11:48:47.308363+0000","last_became_active":"2024-02-18T11:48:46.307253+0000","last_became_peered":"2024-02-18T11:48:46.307253+0000","last_unstale":"2024-02-18T11:48:47.308363+0000","last_undegraded":"2024-02-18T11:48:47.308363+0000","last_fullsized":"2024-02-18T11:48:47.308363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.316738+0000","last_change":"2024-02-18T11:48:47.316738+0000","last_active":"2024-02-18T11:48:47.316738+0000","last_peered":"2024-02-18T11:48:47.316738+0000","last_clean":"2024-02-18T11:48:47.316738+0000","last_became_active":"2024-02-18T11:48:47.316488+0000","last_became_peered":"2024-02-18T11:48:47.316488+0000","last_unstale":"2024-02-18T11:48:47.316738+0000","last_undegraded":"2024-02-18T11:48:47.316738+0000","last_fullsized":"2024-02-18T11:48:47.316738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305690+0000","last_change":"2024-02-18T11:48:46.304153+0000","last_active":"2024-02-18T11:48:47.305690+0000","last_peered":"2024-02-18T11:48:47.305690+0000","last_clean":"2024-02-18T11:48:47.305690+0000","last_became_active":"2024-02-18T11:48:46.303687+0000","last_became_peered":"2024-02-18T11:48:46.303687+0000","last_unstale":"2024-02-18T11:48:47.305690+0000","last_undegraded":"2024-02-18T11:48:47.305690+0000","last_fullsized":"2024-02-18T11:48:47.305690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307009+0000","last_change":"2024-02-18T11:48:47.307009+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307009+0000","last_undegraded":"2024-02-18T11:48:47.307009+0000","last_fullsized":"2024-02-18T11:48:47.307009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304472+0000","last_change":"2024-02-18T11:48:46.301330+0000","last_active":"2024-02-18T11:48:47.304472+0000","last_peered":"2024-02-18T11:48:47.304472+0000","last_clean":"2024-02-18T11:48:47.304472+0000","last_became_active":"2024-02-18T11:48:46.301062+0000","last_became_peered":"2024-02-18T11:48:46.301062+0000","last_unstale":"2024-02-18T11:48:47.304472+0000","last_undegraded":"2024-02-18T11:48:47.304472+0000","last_fullsized":"2024-02-18T11:48:47.304472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309147+0000","last_change":"2024-02-18T11:48:47.309147+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309147+0000","last_undegraded":"2024-02-18T11:48:47.309147+0000","last_fullsized":"2024-02-18T11:48:47.309147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304790+0000","last_change":"2024-02-18T11:48:46.307286+0000","last_active":"2024-02-18T11:48:47.304790+0000","last_peered":"2024-02-18T11:48:47.304790+0000","last_clean":"2024-02-18T11:48:47.304790+0000","last_became_active":"2024-02-18T11:48:46.307002+0000","last_became_peered":"2024-02-18T11:48:46.307002+0000","last_unstale":"2024-02-18T11:48:47.304790+0000","last_undegraded":"2024-02-18T11:48:47.304790+0000","last_fullsized":"2024-02-18T11:48:47.304790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:21.137008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323364+0000","last_change":"2024-02-18T11:48:47.323364+0000","last_active":"2024-02-18T11:48:47.323364+0000","last_peered":"2024-02-18T11:48:47.323364+0000","last_clean":"2024-02-18T11:48:47.323364+0000","last_became_active":"2024-02-18T11:48:47.318734+0000","last_became_peered":"2024-02-18T11:48:47.318734+0000","last_unstale":"2024-02-18T11:48:47.323364+0000","last_undegraded":"2024-02-18T11:48:47.323364+0000","last_fullsized":"2024-02-18T11:48:47.323364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:29:29.908143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306130+0000","last_change":"2024-02-18T11:48:46.305434+0000","last_active":"2024-02-18T11:48:47.306130+0000","last_peered":"2024-02-18T11:48:47.306130+0000","last_clean":"2024-02-18T11:48:47.306130+0000","last_became_active":"2024-02-18T11:48:46.305145+0000","last_became_peered":"2024-02-18T11:48:46.305145+0000","last_unstale":"2024-02-18T11:48:47.306130+0000","last_undegraded":"2024-02-18T11:48:47.306130+0000","last_fullsized":"2024-02-18T11:48:47.306130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:07.775308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308255+0000","last_change":"2024-02-18T11:48:47.308255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308255+0000","last_undegraded":"2024-02-18T11:48:47.308255+0000","last_fullsized":"2024-02-18T11:48:47.308255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301908+0000","last_change":"2024-02-18T11:48:46.297664+0000","last_active":"2024-02-18T11:48:47.301908+0000","last_peered":"2024-02-18T11:48:47.301908+0000","last_clean":"2024-02-18T11:48:47.301908+0000","last_became_active":"2024-02-18T11:48:46.297202+0000","last_became_peered":"2024-02-18T11:48:46.297202+0000","last_unstale":"2024-02-18T11:48:47.301908+0000","last_undegraded":"2024-02-18T11:48:47.301908+0000","last_fullsized":"2024-02-18T11:48:47.301908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.318448+0000","last_change":"2024-02-18T11:48:47.318448+0000","last_active":"2024-02-18T11:48:47.318448+0000","last_peered":"2024-02-18T11:48:47.318448+0000","last_clean":"2024-02-18T11:48:47.318448+0000","last_became_active":"2024-02-18T11:48:47.318190+0000","last_became_peered":"2024-02-18T11:48:47.318190+0000","last_unstale":"2024-02-18T11:48:47.318448+0000","last_undegraded":"2024-02-18T11:48:47.318448+0000","last_fullsized":"2024-02-18T11:48:47.318448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:15:45.332480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306501+0000","last_change":"2024-02-18T11:48:46.300667+0000","last_active":"2024-02-18T11:48:47.306501+0000","last_peered":"2024-02-18T11:48:47.306501+0000","last_clean":"2024-02-18T11:48:47.306501+0000","last_became_active":"2024-02-18T11:48:46.300407+0000","last_became_peered":"2024-02-18T11:48:46.300407+0000","last_unstale":"2024-02-18T11:48:47.306501+0000","last_undegraded":"2024-02-18T11:48:47.306501+0000","last_fullsized":"2024-02-18T11:48:47.306501+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323994+0000","last_change":"2024-02-18T11:48:47.309612+0000","last_active":"2024-02-18T11:48:47.323994+0000","last_peered":"2024-02-18T11:48:47.323994+0000","last_clean":"2024-02-18T11:48:47.323994+0000","last_became_active":"2024-02-18T11:48:47.309382+0000","last_became_peered":"2024-02-18T11:48:47.309382+0000","last_unstale":"2024-02-18T11:48:47.323994+0000","last_undegraded":"2024-02-18T11:48:47.323994+0000","last_fullsized":"2024-02-18T11:48:47.323994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311657+0000","last_change":"2024-02-18T11:48:46.307646+0000","last_active":"2024-02-18T11:48:47.311657+0000","last_peered":"2024-02-18T11:48:47.311657+0000","last_clean":"2024-02-18T11:48:47.311657+0000","last_became_active":"2024-02-18T11:48:46.307345+0000","last_became_peered":"2024-02-18T11:48:46.307345+0000","last_unstale":"2024-02-18T11:48:47.311657+0000","last_undegraded":"2024-02-18T11:48:47.311657+0000","last_fullsized":"2024-02-18T11:48:47.311657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:50.554971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304896+0000","last_change":"2024-02-18T11:48:47.304896+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304896+0000","last_undegraded":"2024-02-18T11:48:47.304896+0000","last_fullsized":"2024-02-18T11:48:47.304896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306767+0000","last_change":"2024-02-18T11:48:46.306743+0000","last_active":"2024-02-18T11:48:47.306767+0000","last_peered":"2024-02-18T11:48:47.306767+0000","last_clean":"2024-02-18T11:48:47.306767+0000","last_became_active":"2024-02-18T11:48:46.306447+0000","last_became_peered":"2024-02-18T11:48:46.306447+0000","last_unstale":"2024-02-18T11:48:47.306767+0000","last_undegraded":"2024-02-18T11:48:47.306767+0000","last_fullsized":"2024-02-18T11:48:47.306767+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.321271+0000","last_change":"2024-02-18T11:48:47.321271+0000","last_active":"2024-02-18T11:48:47.321271+0000","last_peered":"2024-02-18T11:48:47.321271+0000","last_clean":"2024-02-18T11:48:47.321271+0000","last_became_active":"2024-02-18T11:48:47.321021+0000","last_became_peered":"2024-02-18T11:48:47.321021+0000","last_unstale":"2024-02-18T11:48:47.321271+0000","last_undegraded":"2024-02-18T11:48:47.321271+0000","last_fullsized":"2024-02-18T11:48:47.321271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301977+0000","last_change":"2024-02-18T11:48:46.296023+0000","last_active":"2024-02-18T11:48:47.301977+0000","last_peered":"2024-02-18T11:48:47.301977+0000","last_clean":"2024-02-18T11:48:47.301977+0000","last_became_active":"2024-02-18T11:48:46.295715+0000","last_became_peered":"2024-02-18T11:48:46.295715+0000","last_unstale":"2024-02-18T11:48:47.301977+0000","last_undegraded":"2024-02-18T11:48:47.301977+0000","last_fullsized":"2024-02-18T11:48:47.301977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310131+0000","last_change":"2024-02-18T11:48:47.310131+0000","last_active":"2024-02-18T11:48:47.310131+0000","last_peered":"2024-02-18T11:48:47.310131+0000","last_clean":"2024-02-18T11:48:47.310131+0000","last_became_active":"2024-02-18T11:48:47.309877+0000","last_became_peered":"2024-02-18T11:48:47.309877+0000","last_unstale":"2024-02-18T11:48:47.310131+0000","last_undegraded":"2024-02-18T11:48:47.310131+0000","last_fullsized":"2024-02-18T11:48:47.310131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305646+0000","last_change":"2024-02-18T11:48:46.297720+0000","last_active":"2024-02-18T11:48:47.305646+0000","last_peered":"2024-02-18T11:48:47.305646+0000","last_clean":"2024-02-18T11:48:47.305646+0000","last_became_active":"2024-02-18T11:48:46.297462+0000","last_became_peered":"2024-02-18T11:48:46.297462+0000","last_unstale":"2024-02-18T11:48:47.305646+0000","last_undegraded":"2024-02-18T11:48:47.305646+0000","last_fullsized":"2024-02-18T11:48:47.305646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306864+0000","last_change":"2024-02-18T11:48:47.306864+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306864+0000","last_undegraded":"2024-02-18T11:48:47.306864+0000","last_fullsized":"2024-02-18T11:48:47.306864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301800+0000","last_change":"2024-02-18T11:48:46.303136+0000","last_active":"2024-02-18T11:48:47.301800+0000","last_peered":"2024-02-18T11:48:47.301800+0000","last_clean":"2024-02-18T11:48:47.301800+0000","last_became_active":"2024-02-18T11:48:46.302861+0000","last_became_peered":"2024-02-18T11:48:46.302861+0000","last_unstale":"2024-02-18T11:48:47.301800+0000","last_undegraded":"2024-02-18T11:48:47.301800+0000","last_fullsized":"2024-02-18T11:48:47.301800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306944+0000","last_change":"2024-02-18T11:48:47.306944+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306944+0000","last_undegraded":"2024-02-18T11:48:47.306944+0000","last_fullsized":"2024-02-18T11:48:47.306944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.300950+0000","last_change":"2024-02-18T11:48:46.297761+0000","last_active":"2024-02-18T11:48:47.300950+0000","last_peered":"2024-02-18T11:48:47.300950+0000","last_clean":"2024-02-18T11:48:47.300950+0000","last_became_active":"2024-02-18T11:48:46.297500+0000","last_became_peered":"2024-02-18T11:48:46.297500+0000","last_unstale":"2024-02-18T11:48:47.300950+0000","last_undegraded":"2024-02-18T11:48:47.300950+0000","last_fullsized":"2024-02-18T11:48:47.300950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:33:10.032538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310081+0000","last_change":"2024-02-18T11:48:47.310081+0000","last_active":"2024-02-18T11:48:47.310081+0000","last_peered":"2024-02-18T11:48:47.310081+0000","last_clean":"2024-02-18T11:48:47.310081+0000","last_became_active":"2024-02-18T11:48:47.309813+0000","last_became_peered":"2024-02-18T11:48:47.309813+0000","last_unstale":"2024-02-18T11:48:47.310081+0000","last_undegraded":"2024-02-18T11:48:47.310081+0000","last_fullsized":"2024-02-18T11:48:47.310081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306548+0000","last_change":"2024-02-18T11:48:46.298569+0000","last_active":"2024-02-18T11:48:47.306548+0000","last_peered":"2024-02-18T11:48:47.306548+0000","last_clean":"2024-02-18T11:48:47.306548+0000","last_became_active":"2024-02-18T11:48:46.297865+0000","last_became_peered":"2024-02-18T11:48:46.297865+0000","last_unstale":"2024-02-18T11:48:47.306548+0000","last_undegraded":"2024-02-18T11:48:47.306548+0000","last_fullsized":"2024-02-18T11:48:47.306548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:04.067605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.312852+0000","last_change":"2024-02-18T11:48:47.312852+0000","last_active":"2024-02-18T11:48:47.312852+0000","last_peered":"2024-02-18T11:48:47.312852+0000","last_clean":"2024-02-18T11:48:47.312852+0000","last_became_active":"2024-02-18T11:48:47.312584+0000","last_became_peered":"2024-02-18T11:48:47.312584+0000","last_unstale":"2024-02-18T11:48:47.312852+0000","last_undegraded":"2024-02-18T11:48:47.312852+0000","last_fullsized":"2024-02-18T11:48:47.312852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:51.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301208+0000","last_change":"2024-02-18T11:48:46.302429+0000","last_active":"2024-02-18T11:48:47.301208+0000","last_peered":"2024-02-18T11:48:47.301208+0000","last_clean":"2024-02-18T11:48:47.301208+0000","last_became_active":"2024-02-18T11:48:46.302138+0000","last_became_peered":"2024-02-18T11:48:46.302138+0000","last_unstale":"2024-02-18T11:48:47.301208+0000","last_undegraded":"2024-02-18T11:48:47.301208+0000","last_fullsized":"2024-02-18T11:48:47.301208+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:54.091351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305364+0000","last_change":"2024-02-18T11:48:47.305364+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305364+0000","last_undegraded":"2024-02-18T11:48:47.305364+0000","last_fullsized":"2024-02-18T11:48:47.305364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311359+0000","last_change":"2024-02-18T11:48:46.308271+0000","last_active":"2024-02-18T11:48:47.311359+0000","last_peered":"2024-02-18T11:48:47.311359+0000","last_clean":"2024-02-18T11:48:47.311359+0000","last_became_active":"2024-02-18T11:48:46.307981+0000","last_became_peered":"2024-02-18T11:48:46.307981+0000","last_unstale":"2024-02-18T11:48:47.311359+0000","last_undegraded":"2024-02-18T11:48:47.311359+0000","last_fullsized":"2024-02-18T11:48:47.311359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:32.990774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311673+0000","last_change":"2024-02-18T11:48:47.311673+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311673+0000","last_undegraded":"2024-02-18T11:48:47.311673+0000","last_fullsized":"2024-02-18T11:48:47.311673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301714+0000","last_change":"2024-02-18T11:48:46.305551+0000","last_active":"2024-02-18T11:48:47.301714+0000","last_peered":"2024-02-18T11:48:47.301714+0000","last_clean":"2024-02-18T11:48:47.301714+0000","last_became_active":"2024-02-18T11:48:46.305246+0000","last_became_peered":"2024-02-18T11:48:46.305246+0000","last_unstale":"2024-02-18T11:48:47.301714+0000","last_undegraded":"2024-02-18T11:48:47.301714+0000","last_fullsized":"2024-02-18T11:48:47.301714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319083+0000","last_change":"2024-02-18T11:48:47.319083+0000","last_active":"2024-02-18T11:48:47.319083+0000","last_peered":"2024-02-18T11:48:47.319083+0000","last_clean":"2024-02-18T11:48:47.319083+0000","last_became_active":"2024-02-18T11:48:47.318762+0000","last_became_peered":"2024-02-18T11:48:47.318762+0000","last_unstale":"2024-02-18T11:48:47.319083+0000","last_undegraded":"2024-02-18T11:48:47.319083+0000","last_fullsized":"2024-02-18T11:48:47.319083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:55:35.888377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305354+0000","last_change":"2024-02-18T11:48:46.305494+0000","last_active":"2024-02-18T11:48:47.305354+0000","last_peered":"2024-02-18T11:48:47.305354+0000","last_clean":"2024-02-18T11:48:47.305354+0000","last_became_active":"2024-02-18T11:48:46.305222+0000","last_became_peered":"2024-02-18T11:48:46.305222+0000","last_unstale":"2024-02-18T11:48:47.305354+0000","last_undegraded":"2024-02-18T11:48:47.305354+0000","last_fullsized":"2024-02-18T11:48:47.305354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308833+0000","last_change":"2024-02-18T11:48:47.308833+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308833+0000","last_undegraded":"2024-02-18T11:48:47.308833+0000","last_fullsized":"2024-02-18T11:48:47.308833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308558+0000","last_change":"2024-02-18T11:48:46.303722+0000","last_active":"2024-02-18T11:48:47.308558+0000","last_peered":"2024-02-18T11:48:47.308558+0000","last_clean":"2024-02-18T11:48:47.308558+0000","last_became_active":"2024-02-18T11:48:46.303460+0000","last_became_peered":"2024-02-18T11:48:46.303460+0000","last_unstale":"2024-02-18T11:48:47.308558+0000","last_undegraded":"2024-02-18T11:48:47.308558+0000","last_fullsized":"2024-02-18T11:48:47.308558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309759+0000","last_change":"2024-02-18T11:48:47.309759+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309759+0000","last_undegraded":"2024-02-18T11:48:47.309759+0000","last_fullsized":"2024-02-18T11:48:47.309759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.302363+0000","last_change":"2024-02-18T11:48:46.299505+0000","last_active":"2024-02-18T11:48:47.302363+0000","last_peered":"2024-02-18T11:48:47.302363+0000","last_clean":"2024-02-18T11:48:47.302363+0000","last_became_active":"2024-02-18T11:48:46.299230+0000","last_became_peered":"2024-02-18T11:48:46.299230+0000","last_unstale":"2024-02-18T11:48:47.302363+0000","last_undegraded":"2024-02-18T11:48:47.302363+0000","last_fullsized":"2024-02-18T11:48:47.302363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320820+0000","last_change":"2024-02-18T11:48:47.315114+0000","last_active":"2024-02-18T11:48:47.320820+0000","last_peered":"2024-02-18T11:48:47.320820+0000","last_clean":"2024-02-18T11:48:47.320820+0000","last_became_active":"2024-02-18T11:48:47.313064+0000","last_became_peered":"2024-02-18T11:48:47.313064+0000","last_unstale":"2024-02-18T11:48:47.320820+0000","last_undegraded":"2024-02-18T11:48:47.320820+0000","last_fullsized":"2024-02-18T11:48:47.320820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:22.773038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307959+0000","last_change":"2024-02-18T11:48:46.306424+0000","last_active":"2024-02-18T11:48:47.307959+0000","last_peered":"2024-02-18T11:48:47.307959+0000","last_clean":"2024-02-18T11:48:47.307959+0000","last_became_active":"2024-02-18T11:48:46.305938+0000","last_became_peered":"2024-02-18T11:48:46.305938+0000","last_unstale":"2024-02-18T11:48:47.307959+0000","last_undegraded":"2024-02-18T11:48:47.307959+0000","last_fullsized":"2024-02-18T11:48:47.307959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:48.967965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.314001+0000","last_change":"2024-02-18T11:48:47.314001+0000","last_active":"2024-02-18T11:48:47.314001+0000","last_peered":"2024-02-18T11:48:47.314001+0000","last_clean":"2024-02-18T11:48:47.314001+0000","last_became_active":"2024-02-18T11:48:47.313680+0000","last_became_peered":"2024-02-18T11:48:47.313680+0000","last_unstale":"2024-02-18T11:48:47.314001+0000","last_undegraded":"2024-02-18T11:48:47.314001+0000","last_fullsized":"2024-02-18T11:48:47.314001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303228+0000","last_change":"2024-02-18T11:48:46.306107+0000","last_active":"2024-02-18T11:48:47.303228+0000","last_peered":"2024-02-18T11:48:47.303228+0000","last_clean":"2024-02-18T11:48:47.303228+0000","last_became_active":"2024-02-18T11:48:46.305749+0000","last_became_peered":"2024-02-18T11:48:46.305749+0000","last_unstale":"2024-02-18T11:48:47.303228+0000","last_undegraded":"2024-02-18T11:48:47.303228+0000","last_fullsized":"2024-02-18T11:48:47.303228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311076+0000","last_change":"2024-02-18T11:48:47.311076+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311076+0000","last_undegraded":"2024-02-18T11:48:47.311076+0000","last_fullsized":"2024-02-18T11:48:47.311076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307802+0000","last_change":"2024-02-18T11:48:46.303447+0000","last_active":"2024-02-18T11:48:47.307802+0000","last_peered":"2024-02-18T11:48:47.307802+0000","last_clean":"2024-02-18T11:48:47.307802+0000","last_became_active":"2024-02-18T11:48:46.303126+0000","last_became_peered":"2024-02-18T11:48:46.303126+0000","last_unstale":"2024-02-18T11:48:47.307802+0000","last_undegraded":"2024-02-18T11:48:47.307802+0000","last_fullsized":"2024-02-18T11:48:47.307802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:44.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307052+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:48:47.307052+0000","last_peered":"2024-02-18T11:48:47.307052+0000","last_clean":"2024-02-18T11:48:47.307052+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:48:47.307052+0000","last_undegraded":"2024-02-18T11:48:47.307052+0000","last_fullsized":"2024-02-18T11:48:47.307052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21,"ondisk_log_size":21,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:50.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-02-18T11:48:50.938 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305050+0000', 'last_change': '2024-02-18T11:48:47.305050+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305050+0000', 'last_undegraded': '2024-02-18T11:48:47.305050+0000', 'last_fullsized': '2024-02-18T11:48:47.305050+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': ['0', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-02-18T11:48:50.938 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.302121+0000', 'last_change': '2024-02-18T11:48:47.302121+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.302121+0000', 'last_undegraded': '2024-02-18T11:48:47.302121+0000', 'last_fullsized': '2024-02-18T11:48:47.302121+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-02-18T11:48:50.938 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.301388+0000', 'last_change': '2024-02-18T11:48:47.301388+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.301388+0000', 'last_undegraded': '2024-02-18T11:48:47.301388+0000', 'last_fullsized': '2024-02-18T11:48:47.301388+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.313339+0000', 'last_change': '2024-02-18T11:48:47.305848+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.313339+0000', 'last_undegraded': '2024-02-18T11:48:47.313339+0000', 'last_fullsized': '2024-02-18T11:48:47.313339+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.304835+0000', 'last_change': '2024-02-18T11:48:47.304835+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.304835+0000', 'last_undegraded': '2024-02-18T11:48:47.304835+0000', 'last_fullsized': '2024-02-18T11:48:47.304835+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.306155+0000', 'last_change': '2024-02-18T11:48:47.306155+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.306155+0000', 'last_undegraded': '2024-02-18T11:48:47.306155+0000', 'last_fullsized': '2024-02-18T11:48:47.306155+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': ['0', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 6, 'reported_epoch': 37, 'state': 'active', 'last_fresh': '2024-02-18T11:48:47.317646+0000', 'last_change': '2024-02-18T11:48:47.317587+0000', 'last_active': '2024-02-18T11:48:47.317646+0000', 'last_peered': '2024-02-18T11:48:47.317646+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '2024-02-18T11:48:47.317587+0000', 'last_became_peered': '2024-02-18T11:48:47.317587+0000', 'last_unstale': '2024-02-18T11:48:47.317646+0000', 'last_undegraded': '2024-02-18T11:48:47.317646+0000', 'last_fullsized': '2024-02-18T11:48:47.317646+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-02-18T11:48:50.939 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305372+0000', 'last_change': '2024-02-18T11:48:47.305372+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305372+0000', 'last_undegraded': '2024-02-18T11:48:47.305372+0000', 'last_fullsized': '2024-02-18T11:48:47.305372+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305270+0000', 'last_change': '2024-02-18T11:48:47.305270+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305270+0000', 'last_undegraded': '2024-02-18T11:48:47.305270+0000', 'last_fullsized': '2024-02-18T11:48:47.305270+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305144+0000', 'last_change': '2024-02-18T11:48:47.305144+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305144+0000', 'last_undegraded': '2024-02-18T11:48:47.305144+0000', 'last_fullsized': '2024-02-18T11:48:47.305144+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.304457+0000', 'last_change': '2024-02-18T11:48:47.304457+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.304457+0000', 'last_undegraded': '2024-02-18T11:48:47.304457+0000', 'last_fullsized': '2024-02-18T11:48:47.304457+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': ['5', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305255+0000', 'last_change': '2024-02-18T11:48:47.305255+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305255+0000', 'last_undegraded': '2024-02-18T11:48:47.305255+0000', 'last_fullsized': '2024-02-18T11:48:47.305255+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.940 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.308258+0000', 'last_change': '2024-02-18T11:48:47.308258+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.308258+0000', 'last_undegraded': '2024-02-18T11:48:47.308258+0000', 'last_fullsized': '2024-02-18T11:48:47.308258+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.301260+0000', 'last_change': '2024-02-18T11:48:47.301260+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.301260+0000', 'last_undegraded': '2024-02-18T11:48:47.301260+0000', 'last_fullsized': '2024-02-18T11:48:47.301260+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.301870+0000', 'last_change': '2024-02-18T11:48:47.301870+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.301870+0000', 'last_undegraded': '2024-02-18T11:48:47.301870+0000', 'last_fullsized': '2024-02-18T11:48:47.301870+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.301207+0000', 'last_change': '2024-02-18T11:48:47.301207+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.301207+0000', 'last_undegraded': '2024-02-18T11:48:47.301207+0000', 'last_fullsized': '2024-02-18T11:48:47.301207+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305500+0000', 'last_change': '2024-02-18T11:48:47.305500+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305500+0000', 'last_undegraded': '2024-02-18T11:48:47.305500+0000', 'last_fullsized': '2024-02-18T11:48:47.305500+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-02-18T11:48:50.941 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.304586+0000', 'last_change': '2024-02-18T11:48:47.304586+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.304586+0000', 'last_undegraded': '2024-02-18T11:48:47.304586+0000', 'last_fullsized': '2024-02-18T11:48:47.304586+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305254+0000', 'last_change': '2024-02-18T11:48:47.305254+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305254+0000', 'last_undegraded': '2024-02-18T11:48:47.305254+0000', 'last_fullsized': '2024-02-18T11:48:47.305254+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.300930+0000', 'last_change': '2024-02-18T11:48:47.300930+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.300930+0000', 'last_undegraded': '2024-02-18T11:48:47.300930+0000', 'last_fullsized': '2024-02-18T11:48:47.300930+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': ['2', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 6, 'reported_epoch': 37, 'state': 'active', 'last_fresh': '2024-02-18T11:48:47.315231+0000', 'last_change': '2024-02-18T11:48:47.315173+0000', 'last_active': '2024-02-18T11:48:47.315231+0000', 'last_peered': '2024-02-18T11:48:47.315231+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '2024-02-18T11:48:47.315173+0000', 'last_became_peered': '2024-02-18T11:48:47.315173+0000', 'last_unstale': '2024-02-18T11:48:47.315231+0000', 'last_undegraded': '2024-02-18T11:48:47.315231+0000', 'last_fullsized': '2024-02-18T11:48:47.315231+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.310414+0000', 'last_change': '2024-02-18T11:48:47.310414+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.310414+0000', 'last_undegraded': '2024-02-18T11:48:47.310414+0000', 'last_fullsized': '2024-02-18T11:48:47.310414+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.306092+0000', 'last_change': '2024-02-18T11:48:47.306092+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.306092+0000', 'last_undegraded': '2024-02-18T11:48:47.306092+0000', 'last_fullsized': '2024-02-18T11:48:47.306092+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.301304+0000', 'last_change': '2024-02-18T11:48:47.301304+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.301304+0000', 'last_undegraded': '2024-02-18T11:48:47.301304+0000', 'last_fullsized': '2024-02-18T11:48:47.301304+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': ['6', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305634+0000', 'last_change': '2024-02-18T11:48:47.305634+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305634+0000', 'last_undegraded': '2024-02-18T11:48:47.305634+0000', 'last_fullsized': '2024-02-18T11:48:47.305634+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305480+0000', 'last_change': '2024-02-18T11:48:47.305480+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305480+0000', 'last_undegraded': '2024-02-18T11:48:47.305480+0000', 'last_fullsized': '2024-02-18T11:48:47.305480+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': ['1', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.307947+0000', 'last_change': '2024-02-18T11:48:47.307947+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.307947+0000', 'last_undegraded': '2024-02-18T11:48:47.307947+0000', 'last_fullsized': '2024-02-18T11:48:47.307947+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': ['7', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.306066+0000', 'last_change': '2024-02-18T11:48:47.306066+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.306066+0000', 'last_undegraded': '2024-02-18T11:48:47.306066+0000', 'last_fullsized': '2024-02-18T11:48:47.306066+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-02-18T11:48:50.943 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.309454+0000', 'last_change': '2024-02-18T11:48:47.309454+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.309454+0000', 'last_undegraded': '2024-02-18T11:48:47.309454+0000', 'last_fullsized': '2024-02-18T11:48:47.309454+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305889+0000', 'last_change': '2024-02-18T11:48:47.305889+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305889+0000', 'last_undegraded': '2024-02-18T11:48:47.305889+0000', 'last_fullsized': '2024-02-18T11:48:47.305889+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.301870+0000', 'last_change': '2024-02-18T11:48:47.301870+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.301870+0000', 'last_undegraded': '2024-02-18T11:48:47.301870+0000', 'last_fullsized': '2024-02-18T11:48:47.301870+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.311877+0000', 'last_change': '2024-02-18T11:48:47.311877+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.311877+0000', 'last_undegraded': '2024-02-18T11:48:47.311877+0000', 'last_fullsized': '2024-02-18T11:48:47.311877+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.307009+0000', 'last_change': '2024-02-18T11:48:47.307009+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.307009+0000', 'last_undegraded': '2024-02-18T11:48:47.307009+0000', 'last_fullsized': '2024-02-18T11:48:47.307009+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.309147+0000', 'last_change': '2024-02-18T11:48:47.309147+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.309147+0000', 'last_undegraded': '2024-02-18T11:48:47.309147+0000', 'last_fullsized': '2024-02-18T11:48:47.309147+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': ['0', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-02-18T11:48:50.944 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.308255+0000', 'last_change': '2024-02-18T11:48:47.308255+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.308255+0000', 'last_undegraded': '2024-02-18T11:48:47.308255+0000', 'last_fullsized': '2024-02-18T11:48:47.308255+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.304896+0000', 'last_change': '2024-02-18T11:48:47.304896+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.304896+0000', 'last_undegraded': '2024-02-18T11:48:47.304896+0000', 'last_fullsized': '2024-02-18T11:48:47.304896+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.306864+0000', 'last_change': '2024-02-18T11:48:47.306864+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.306864+0000', 'last_undegraded': '2024-02-18T11:48:47.306864+0000', 'last_fullsized': '2024-02-18T11:48:47.306864+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.306944+0000', 'last_change': '2024-02-18T11:48:47.306944+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.306944+0000', 'last_undegraded': '2024-02-18T11:48:47.306944+0000', 'last_fullsized': '2024-02-18T11:48:47.306944+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.305364+0000', 'last_change': '2024-02-18T11:48:47.305364+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.305364+0000', 'last_undegraded': '2024-02-18T11:48:47.305364+0000', 'last_fullsized': '2024-02-18T11:48:47.305364+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-02-18T11:48:50.945 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.311673+0000', 'last_change': '2024-02-18T11:48:47.311673+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.311673+0000', 'last_undegraded': '2024-02-18T11:48:47.311673+0000', 'last_fullsized': '2024-02-18T11:48:47.311673+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-02-18T11:48:50.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-02-18T11:48:50.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.308833+0000', 'last_change': '2024-02-18T11:48:47.308833+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.308833+0000', 'last_undegraded': '2024-02-18T11:48:47.308833+0000', 'last_fullsized': '2024-02-18T11:48:47.308833+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-02-18T11:48:50.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-02-18T11:48:50.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.309759+0000', 'last_change': '2024-02-18T11:48:47.309759+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.309759+0000', 'last_undegraded': '2024-02-18T11:48:47.309759+0000', 'last_fullsized': '2024-02-18T11:48:47.309759+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': ['1', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-02-18T11:48:50.946 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-02-18T11:48:50.946 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-02-18T11:48:47.311076+0000', 'last_change': '2024-02-18T11:48:47.311076+0000', 'last_active': '2024-02-18T11:48:46.281573+0000', 'last_peered': '2024-02-18T11:48:46.281573+0000', 'last_clean': '2024-02-18T11:48:46.281573+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-02-18T11:48:47.311076+0000', 'last_undegraded': '2024-02-18T11:48:47.311076+0000', 'last_fullsized': '2024-02-18T11:48:47.311076+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'last_clean_scrub_stamp': '2024-02-18T11:48:46.281573+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-02-18T11:48:50.947 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-02-18T11:48:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 Processor -- start 2024-02-18T11:48:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 -- start start 2024-02-18T11:48:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5cf806c1f0 0x7f5cf815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cf80ad740 0x7f5cf806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf80af870 con 0x7f5cf815c9c0 2024-02-18T11:48:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf806c5d0 con 0x7f5cf806c1f0 2024-02-18T11:48:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.081+0000 7f5cfd113640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf815c530 con 0x7f5cf80ad740 2024-02-18T11:48:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf6d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5cf806c1f0 0x7f5cf815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf659c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5cf815c9c0 0x7f5cf80ad030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34758/0 (socket says 172.21.15.116:34758) 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf659c640 1 -- 172.21.15.116:0/3958686879 learned_addr learned my addr 172.21.15.116:0/3958686879 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf759e640 1 --2- 172.21.15.116:0/3958686879 >> v2:172.21.15.143:3300/0 conn(0x7f5cf80ad740 0x7f5cf806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf6d9d640 1 -- 172.21.15.116:0/3958686879 >> v2:172.21.15.143:3300/0 conn(0x7f5cf80ad740 msgr2=0x7f5cf806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf6d9d640 1 --2- 172.21.15.116:0/3958686879 >> v2:172.21.15.143:3300/0 conn(0x7f5cf80ad740 0x7f5cf806ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf6d9d640 1 -- 172.21.15.116:0/3958686879 >> v1:172.21.15.116:6789/0 conn(0x7f5cf815c9c0 legacy=0x7f5cf80ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.076 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.082+0000 7f5cf6d9d640 1 -- 172.21.15.116:0/3958686879 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8160280 con 0x7f5cf806c1f0 2024-02-18T11:48:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cf6d9d640 1 --2- 172.21.15.116:0/3958686879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf806c1f0 0x7f5cf815fd40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8156390 tx=0x7f5cf002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e919a25382fb5b server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cf5d9b640 1 -- 172.21.15.116:0/3958686879 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf002cbf0 con 0x7f5cf806c1f0 2024-02-18T11:48:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cf5d9b640 1 -- 172.21.15.116:0/3958686879 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf002cd50 con 0x7f5cf806c1f0 2024-02-18T11:48:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cf5d9b640 1 -- 172.21.15.116:0/3958686879 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0034660 con 0x7f5cf806c1f0 2024-02-18T11:48:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 -- 172.21.15.116:0/3958686879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf806c1f0 msgr2=0x7f5cf815fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 --2- 172.21.15.116:0/3958686879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf806c1f0 0x7f5cf815fd40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8156390 tx=0x7f5cf002be40 comp rx=0 tx=0).stop 2024-02-18T11:48:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 -- 172.21.15.116:0/3958686879 shutdown_connections 2024-02-18T11:48:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 --2- 172.21.15.116:0/3958686879 >> v2:172.21.15.143:3300/0 conn(0x7f5cf80ad740 0x7f5cf806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 --2- 172.21.15.116:0/3958686879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf806c1f0 0x7f5cf815fd40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 -- 172.21.15.116:0/3958686879 >> 172.21.15.116:0/3958686879 conn(0x7f5cf80b23d0 msgr2=0x7f5cf80cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 -- 172.21.15.116:0/3958686879 shutdown_connections 2024-02-18T11:48:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.083+0000 7f5cfd113640 1 -- 172.21.15.116:0/3958686879 wait complete. 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 Processor -- start 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 -- start start 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cf806c1f0 0x7f5cf8147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cf80ad740 0x7f5cf80cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 0x7f5cf80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf8147da0 con 0x7f5cf815c9c0 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf80ce3f0 con 0x7f5cf806c1f0 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cfd113640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf81f45d0 con 0x7f5cf80ad740 2024-02-18T11:48:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.085+0000 7f5cf759e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 0x7f5cf80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 0x7f5cf80b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42000/0 (socket says 172.21.15.116:42000) 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 -- 172.21.15.116:0/1259199299 learned_addr learned my addr 172.21.15.116:0/1259199299 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf659c640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cf80ad740 0x7f5cf80cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf6d9d640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cf806c1f0 0x7f5cf8147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 -- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cf80ad740 msgr2=0x7f5cf80cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cf80ad740 0x7f5cf80cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 -- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cf806c1f0 msgr2=0x7f5cf8147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cf806c1f0 0x7f5cf8147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 -- 172.21.15.116:0/1259199299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009a70 con 0x7f5cf815c9c0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cf759e640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 0x7f5cf80b4770 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800d600 tx=0x7f5ce800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80125f0 con 0x7f5cf815c9c0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce8012750 con 0x7f5cf815c9c0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf81f3a50 con 0x7f5cf815c9c0 2024-02-18T11:48:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8012b00 con 0x7f5cf815c9c0 2024-02-18T11:48:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.086+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf81f4010 con 0x7f5cf815c9c0 2024-02-18T11:48:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.087+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf80b2f00 con 0x7f5cf815c9c0 2024-02-18T11:48:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.088+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f5ce8012c60 con 0x7f5cf815c9c0 2024-02-18T11:48:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.088+0000 7f5cef7fe640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5cd806bc70 0x7f5cd806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.088+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ce808acf0 con 0x7f5cf815c9c0 2024-02-18T11:48:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.088+0000 7f5cf6d9d640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5cd806bc70 0x7f5cd806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.083 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.088+0000 7f5cf6d9d640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5cd806bc70 0x7f5cd806e130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0036000 tx=0x7f5cf0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:51.085 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.092+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ce808f070 con 0x7f5cf815c9c0 2024-02-18T11:48:51.165 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T11:48:51.172+0000 7f30ed773640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-18T11:48:51.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.331+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5cf8074520 con 0x7f5cf815c9c0 2024-02-18T11:48:51.325 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.332+0000 7f5cef7fe640 1 -- 172.21.15.116:0/1259199299 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1136 (secure 0 0 0) 0x7f5ce80553f0 con 0x7f5cf815c9c0 2024-02-18T11:48:51.325 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:51.325 INFO:teuthology.orchestra.run.smithi116.stdout:{"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":101,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":38,"num_osds":8,"num_up_osds":8,"osd_up_since":1708256841,"num_in_osds":8,"osd_in_since":1708256832,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":54},{"state_name":"creating+activating","count":41},{"state_name":"active","count":2}],"num_pgs":97,"num_pools":3,"num_objects":14,"data_bytes":591656,"bytes_used":225882112,"bytes_avail":772868231168,"bytes_total":773094113280,"inactive_pgs_ratio":0.42268040776252747,"write_bytes_sec":2038,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5358}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-02-18T11:48:36.304061+0000","services":{}},"progress_events":{}} 2024-02-18T11:48:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5cd806bc70 msgr2=0x7f5cd806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5cd806bc70 0x7f5cd806e130 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0036000 tx=0x7f5cf0030000 comp rx=0 tx=0).stop 2024-02-18T11:48:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 msgr2=0x7f5cf80b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 0x7f5cf80b4770 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800d600 tx=0x7f5ce800dac0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 shutdown_connections 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f5cd806bc70 0x7f5cd806e130 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cf815c9c0 0x7f5cf80b4770 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cf80ad740 0x7f5cf80cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 --2- 172.21.15.116:0/1259199299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cf806c1f0 0x7f5cf8147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 >> 172.21.15.116:0/1259199299 conn(0x7f5cf80b23d0 msgr2=0x7f5cf815d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 shutdown_connections 2024-02-18T11:48:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.334+0000 7f5cfd113640 1 -- 172.21.15.116:0/1259199299 wait complete. 2024-02-18T11:48:51.338 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-02-18T11:48:51.338 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:51.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.445+0000 7f36db657640 1 Processor -- start 2024-02-18T11:48:51.439 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.446+0000 7f36db657640 1 -- start start 2024-02-18T11:48:51.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.446+0000 7f36db657640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36d40b0210 0x7f36d40b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.446+0000 7f36db657640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d415ae40 0x7f36d40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36db657640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d40cdff0 con 0x7f36d40cefe0 2024-02-18T11:48:51.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36db657640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d40b1070 con 0x7f36d40b0210 2024-02-18T11:48:51.440 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36db657640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d40b1ea0 con 0x7f36d415ae40 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36d40b0210 0x7f36d40b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36d40b0210 0x7f36d40b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42008/0 (socket says 172.21.15.116:42008) 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 -- 172.21.15.116:0/1220960345 learned_addr learned my addr 172.21.15.116:0/1220960345 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 -- 172.21.15.116:0/1220960345 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d415ae40 msgr2=0x7f36d40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 --2- 172.21.15.116:0/1220960345 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d415ae40 0x7f36d40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 -- 172.21.15.116:0/1220960345 >> v1:172.21.15.143:6789/0 conn(0x7f36d40cefe0 legacy=0x7f36d406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:51.441 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.447+0000 7f36d93f3640 1 -- 172.21.15.116:0/1220960345 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d415b3e0 con 0x7f36d40b0210 2024-02-18T11:48:51.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36d93f3640 1 --2- 172.21.15.116:0/1220960345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d40b1960 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009870 tx=0x7f36c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64161073b8cd8acb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:51.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36c7fff640 1 -- 172.21.15.116:0/1220960345 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8031640 con 0x7f36d40b0210 2024-02-18T11:48:51.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36c7fff640 1 -- 172.21.15.116:0/1220960345 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c80317a0 con 0x7f36d40b0210 2024-02-18T11:48:51.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36c7fff640 1 -- 172.21.15.116:0/1220960345 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8031b30 con 0x7f36d40b0210 2024-02-18T11:48:51.442 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36db657640 1 -- 172.21.15.116:0/1220960345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 msgr2=0x7f36d40b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36db657640 1 --2- 172.21.15.116:0/1220960345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d40b1960 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009870 tx=0x7f36c802b8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36db657640 1 -- 172.21.15.116:0/1220960345 shutdown_connections 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36db657640 1 --2- 172.21.15.116:0/1220960345 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d415ae40 0x7f36d40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36db657640 1 --2- 172.21.15.116:0/1220960345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d40b1960 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.448+0000 7f36db657640 1 -- 172.21.15.116:0/1220960345 >> 172.21.15.116:0/1220960345 conn(0x7f36d4066f20 msgr2=0x7f36d406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.449+0000 7f36db657640 1 -- 172.21.15.116:0/1220960345 shutdown_connections 2024-02-18T11:48:51.443 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.449+0000 7f36db657640 1 -- 172.21.15.116:0/1220960345 wait complete. 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 Processor -- start 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 -- start start 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d41569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d40cefe0 0x7f36d413a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d415ae40 0x7f36d415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4157cd0 con 0x7f36d40b0210 2024-02-18T11:48:51.445 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d414f600 con 0x7f36d415ae40 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36db657640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d41570c0 con 0x7f36d40cefe0 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36d93f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d41569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36d93f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d41569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42014/0 (socket says 172.21.15.116:42014) 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.450+0000 7f36d93f3640 1 -- 172.21.15.116:0/772821998 learned_addr learned my addr 172.21.15.116:0/772821998 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d8bf2640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d40cefe0 0x7f36d413a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d9bf4640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d415ae40 0x7f36d415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d93f3640 1 -- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d40cefe0 msgr2=0x7f36d413a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d93f3640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d40cefe0 0x7f36d413a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d93f3640 1 -- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d415ae40 msgr2=0x7f36d415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d93f3640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d415ae40 0x7f36d415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d93f3640 1 -- 172.21.15.116:0/772821998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80094d0 con 0x7f36d40b0210 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36d93f3640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d41569f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009870 tx=0x7f36c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c803ac60 con 0x7f36d40b0210 2024-02-18T11:48:51.446 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c803adc0 con 0x7f36d40b0210 2024-02-18T11:48:51.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d41573a0 con 0x7f36d40b0210 2024-02-18T11:48:51.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8044630 con 0x7f36d40b0210 2024-02-18T11:48:51.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.451+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d4157960 con 0x7f36d40b0210 2024-02-18T11:48:51.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.452+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d4074520 con 0x7f36d40b0210 2024-02-18T11:48:51.447 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.453+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f36c802f050 con 0x7f36d40b0210 2024-02-18T11:48:51.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.453+0000 7f36c5ffb640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f36b806bc70 0x7f36b806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:51.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.453+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f36c80ac8d0 con 0x7f36d40b0210 2024-02-18T11:48:51.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.453+0000 7f36d8bf2640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f36b806bc70 0x7f36b806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:51.448 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.454+0000 7f36d8bf2640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f36b806bc70 0x7f36b806e130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36d4150800 tx=0x7f36d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:51.450 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.458+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36c8076f20 con 0x7f36d40b0210 2024-02-18T11:48:51.616 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.623+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36d40ce220 con 0x7f36b806bc70 2024-02-18T11:48:51.622 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.629+0000 7f36c5ffb640 1 -- 172.21.15.116:0/772821998 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248905 (secure 0 0 0) 0x7f36d0009480 con 0x7f36b806bc70 2024-02-18T11:48:51.625 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:51.626 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:51.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f36b806bc70 msgr2=0x7f36b806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f36b806bc70 0x7f36b806e130 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f36d4150800 tx=0x7f36d0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:51.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 msgr2=0x7f36d41569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:51.627 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d41569f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009870 tx=0x7f36c80046c0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 shutdown_connections 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f36b806bc70 0x7f36b806e130 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36d415ae40 0x7f36d415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f36d40cefe0 0x7f36d413a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 --2- 172.21.15.116:0/772821998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36d40b0210 0x7f36d41569f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 >> 172.21.15.116:0/772821998 conn(0x7f36d4066f20 msgr2=0x7f36d4159480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 shutdown_connections 2024-02-18T11:48:51.628 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:51.632+0000 7f36db657640 1 -- 172.21.15.116:0/772821998 wait complete. 2024-02-18T11:48:51.636 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-02-18T11:48:50.306500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591656,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220588,"kb_used_data":4404,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754132,"statfs":{"total":773094113280,"available":772868231168,"internally_reserved":0,"allocated":4509696,"data_stored":2598551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.024513"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319567+0000","last_change":"2024-02-18T11:48:47.319567+0000","last_active":"2024-02-18T11:48:47.319567+0000","last_peered":"2024-02-18T11:48:47.319567+0000","last_clean":"2024-02-18T11:48:47.319567+0000","last_became_active":"2024-02-18T11:48:47.318466+0000","last_became_peered":"2024-02-18T11:48:47.318466+0000","last_unstale":"2024-02-18T11:48:47.319567+0000","last_undegraded":"2024-02-18T11:48:47.319567+0000","last_fullsized":"2024-02-18T11:48:47.319567+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:35:02.679276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317654+0000","last_change":"2024-02-18T11:48:47.317654+0000","last_active":"2024-02-18T11:48:47.317654+0000","last_peered":"2024-02-18T11:48:47.317654+0000","last_clean":"2024-02-18T11:48:47.317654+0000","last_became_active":"2024-02-18T11:48:47.316735+0000","last_became_peered":"2024-02-18T11:48:47.316735+0000","last_unstale":"2024-02-18T11:48:47.317654+0000","last_undegraded":"2024-02-18T11:48:47.317654+0000","last_fullsized":"2024-02-18T11:48:47.317654+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305050+0000","last_change":"2024-02-18T11:48:47.305050+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305050+0000","last_undegraded":"2024-02-18T11:48:47.305050+0000","last_fullsized":"2024-02-18T11:48:47.305050+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.302121+0000","last_change":"2024-02-18T11:48:47.302121+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.302121+0000","last_undegraded":"2024-02-18T11:48:47.302121+0000","last_fullsized":"2024-02-18T11:48:47.302121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301388+0000","last_change":"2024-02-18T11:48:47.301388+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301388+0000","last_undegraded":"2024-02-18T11:48:47.301388+0000","last_fullsized":"2024-02-18T11:48:47.301388+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":4,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.313339+0000","last_change":"2024-02-18T11:48:47.305848+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.313339+0000","last_undegraded":"2024-02-18T11:48:47.313339+0000","last_fullsized":"2024-02-18T11:48:47.313339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304835+0000","last_change":"2024-02-18T11:48:47.304835+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304835+0000","last_undegraded":"2024-02-18T11:48:47.304835+0000","last_fullsized":"2024-02-18T11:48:47.304835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319168+0000","last_change":"2024-02-18T11:48:47.319168+0000","last_active":"2024-02-18T11:48:47.319168+0000","last_peered":"2024-02-18T11:48:47.319168+0000","last_clean":"2024-02-18T11:48:47.319168+0000","last_became_active":"2024-02-18T11:48:47.318918+0000","last_became_peered":"2024-02-18T11:48:47.318918+0000","last_unstale":"2024-02-18T11:48:47.319168+0000","last_undegraded":"2024-02-18T11:48:47.319168+0000","last_fullsized":"2024-02-18T11:48:47.319168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:03:08.526820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306155+0000","last_change":"2024-02-18T11:48:47.306155+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306155+0000","last_undegraded":"2024-02-18T11:48:47.306155+0000","last_fullsized":"2024-02-18T11:48:47.306155+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317587+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.317587+0000","last_became_peered":"2024-02-18T11:48:47.317587+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305372+0000","last_change":"2024-02-18T11:48:47.305372+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305372+0000","last_undegraded":"2024-02-18T11:48:47.305372+0000","last_fullsized":"2024-02-18T11:48:47.305372+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305270+0000","last_change":"2024-02-18T11:48:47.305270+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305270+0000","last_undegraded":"2024-02-18T11:48:47.305270+0000","last_fullsized":"2024-02-18T11:48:47.305270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305144+0000","last_change":"2024-02-18T11:48:47.305144+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305144+0000","last_undegraded":"2024-02-18T11:48:47.305144+0000","last_fullsized":"2024-02-18T11:48:47.305144+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320799+0000","last_change":"2024-02-18T11:48:47.314842+0000","last_active":"2024-02-18T11:48:47.320799+0000","last_peered":"2024-02-18T11:48:47.320799+0000","last_clean":"2024-02-18T11:48:47.320799+0000","last_became_active":"2024-02-18T11:48:47.312762+0000","last_became_peered":"2024-02-18T11:48:47.312762+0000","last_unstale":"2024-02-18T11:48:47.320799+0000","last_undegraded":"2024-02-18T11:48:47.320799+0000","last_fullsized":"2024-02-18T11:48:47.320799+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:08.779058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304457+0000","last_change":"2024-02-18T11:48:47.304457+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304457+0000","last_undegraded":"2024-02-18T11:48:47.304457+0000","last_fullsized":"2024-02-18T11:48:47.304457+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305255+0000","last_change":"2024-02-18T11:48:47.305255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305255+0000","last_undegraded":"2024-02-18T11:48:47.305255+0000","last_fullsized":"2024-02-18T11:48:47.305255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308258+0000","last_change":"2024-02-18T11:48:47.308258+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308258+0000","last_undegraded":"2024-02-18T11:48:47.308258+0000","last_fullsized":"2024-02-18T11:48:47.308258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301260+0000","last_change":"2024-02-18T11:48:47.301260+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301260+0000","last_undegraded":"2024-02-18T11:48:47.301260+0000","last_fullsized":"2024-02-18T11:48:47.301260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301207+0000","last_change":"2024-02-18T11:48:47.301207+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301207+0000","last_undegraded":"2024-02-18T11:48:47.301207+0000","last_fullsized":"2024-02-18T11:48:47.301207+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305500+0000","last_change":"2024-02-18T11:48:47.305500+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305500+0000","last_undegraded":"2024-02-18T11:48:47.305500+0000","last_fullsized":"2024-02-18T11:48:47.305500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304586+0000","last_change":"2024-02-18T11:48:47.304586+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304586+0000","last_undegraded":"2024-02-18T11:48:47.304586+0000","last_fullsized":"2024-02-18T11:48:47.304586+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305254+0000","last_change":"2024-02-18T11:48:47.305254+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305254+0000","last_undegraded":"2024-02-18T11:48:47.305254+0000","last_fullsized":"2024-02-18T11:48:47.305254+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.300930+0000","last_change":"2024-02-18T11:48:47.300930+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.300930+0000","last_undegraded":"2024-02-18T11:48:47.300930+0000","last_fullsized":"2024-02-18T11:48:47.300930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":6,"reported_epoch":37,"state":"active","last_fresh":"2024-02-18T11:48:47.315231+0000","last_change":"2024-02-18T11:48:47.315173+0000","last_active":"2024-02-18T11:48:47.315231+0000","last_peered":"2024-02-18T11:48:47.315231+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"2024-02-18T11:48:47.315173+0000","last_became_peered":"2024-02-18T11:48:47.315173+0000","last_unstale":"2024-02-18T11:48:47.315231+0000","last_undegraded":"2024-02-18T11:48:47.315231+0000","last_fullsized":"2024-02-18T11:48:47.315231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.310414+0000","last_change":"2024-02-18T11:48:47.310414+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.310414+0000","last_undegraded":"2024-02-18T11:48:47.310414+0000","last_fullsized":"2024-02-18T11:48:47.310414+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310027+0000","last_change":"2024-02-18T11:48:47.309300+0000","last_active":"2024-02-18T11:48:47.310027+0000","last_peered":"2024-02-18T11:48:47.310027+0000","last_clean":"2024-02-18T11:48:47.310027+0000","last_became_active":"2024-02-18T11:48:47.309025+0000","last_became_peered":"2024-02-18T11:48:47.309025+0000","last_unstale":"2024-02-18T11:48:47.310027+0000","last_undegraded":"2024-02-18T11:48:47.310027+0000","last_fullsized":"2024-02-18T11:48:47.310027+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306092+0000","last_change":"2024-02-18T11:48:47.306092+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306092+0000","last_undegraded":"2024-02-18T11:48:47.306092+0000","last_fullsized":"2024-02-18T11:48:47.306092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301304+0000","last_change":"2024-02-18T11:48:47.301304+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301304+0000","last_undegraded":"2024-02-18T11:48:47.301304+0000","last_fullsized":"2024-02-18T11:48:47.301304+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311897+0000","last_change":"2024-02-18T11:48:47.311897+0000","last_active":"2024-02-18T11:48:47.311897+0000","last_peered":"2024-02-18T11:48:47.311897+0000","last_clean":"2024-02-18T11:48:47.311897+0000","last_became_active":"2024-02-18T11:48:47.311571+0000","last_became_peered":"2024-02-18T11:48:47.311571+0000","last_unstale":"2024-02-18T11:48:47.311897+0000","last_undegraded":"2024-02-18T11:48:47.311897+0000","last_fullsized":"2024-02-18T11:48:47.311897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:46.613859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305634+0000","last_change":"2024-02-18T11:48:47.305634+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305634+0000","last_undegraded":"2024-02-18T11:48:47.305634+0000","last_fullsized":"2024-02-18T11:48:47.305634+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305480+0000","last_change":"2024-02-18T11:48:47.305480+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305480+0000","last_undegraded":"2024-02-18T11:48:47.305480+0000","last_fullsized":"2024-02-18T11:48:47.305480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307947+0000","last_change":"2024-02-18T11:48:47.307947+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307947+0000","last_undegraded":"2024-02-18T11:48:47.307947+0000","last_fullsized":"2024-02-18T11:48:47.307947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":["7","5"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'3","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301842+0000","last_change":"2024-02-18T11:48:46.297461+0000","last_active":"2024-02-18T11:48:47.301842+0000","last_peered":"2024-02-18T11:48:47.301842+0000","last_clean":"2024-02-18T11:48:47.301842+0000","last_became_active":"2024-02-18T11:48:46.297166+0000","last_became_peered":"2024-02-18T11:48:46.297166+0000","last_unstale":"2024-02-18T11:48:47.301842+0000","last_undegraded":"2024-02-18T11:48:47.301842+0000","last_fullsized":"2024-02-18T11:48:47.301842+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306066+0000","last_change":"2024-02-18T11:48:47.306066+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306066+0000","last_undegraded":"2024-02-18T11:48:47.306066+0000","last_fullsized":"2024-02-18T11:48:47.306066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305368+0000","last_change":"2024-02-18T11:48:46.300302+0000","last_active":"2024-02-18T11:48:47.305368+0000","last_peered":"2024-02-18T11:48:47.305368+0000","last_clean":"2024-02-18T11:48:47.305368+0000","last_became_active":"2024-02-18T11:48:46.299909+0000","last_became_peered":"2024-02-18T11:48:46.299909+0000","last_unstale":"2024-02-18T11:48:47.305368+0000","last_undegraded":"2024-02-18T11:48:47.305368+0000","last_fullsized":"2024-02-18T11:48:47.305368+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319795+0000","last_change":"2024-02-18T11:48:47.316579+0000","last_active":"2024-02-18T11:48:47.319795+0000","last_peered":"2024-02-18T11:48:47.319795+0000","last_clean":"2024-02-18T11:48:47.319795+0000","last_became_active":"2024-02-18T11:48:47.316195+0000","last_became_peered":"2024-02-18T11:48:47.316195+0000","last_unstale":"2024-02-18T11:48:47.319795+0000","last_undegraded":"2024-02-18T11:48:47.319795+0000","last_fullsized":"2024-02-18T11:48:47.319795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:18.861179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310566+0000","last_change":"2024-02-18T11:48:46.297756+0000","last_active":"2024-02-18T11:48:47.310566+0000","last_peered":"2024-02-18T11:48:47.310566+0000","last_clean":"2024-02-18T11:48:47.310566+0000","last_became_active":"2024-02-18T11:48:46.297485+0000","last_became_peered":"2024-02-18T11:48:46.297485+0000","last_unstale":"2024-02-18T11:48:47.310566+0000","last_undegraded":"2024-02-18T11:48:47.310566+0000","last_fullsized":"2024-02-18T11:48:47.310566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317072+0000","last_change":"2024-02-18T11:48:47.317072+0000","last_active":"2024-02-18T11:48:47.317072+0000","last_peered":"2024-02-18T11:48:47.317072+0000","last_clean":"2024-02-18T11:48:47.317072+0000","last_became_active":"2024-02-18T11:48:47.316360+0000","last_became_peered":"2024-02-18T11:48:47.316360+0000","last_unstale":"2024-02-18T11:48:47.317072+0000","last_undegraded":"2024-02-18T11:48:47.317072+0000","last_fullsized":"2024-02-18T11:48:47.317072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:08:26.674078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310420+0000","last_change":"2024-02-18T11:48:46.305539+0000","last_active":"2024-02-18T11:48:47.310420+0000","last_peered":"2024-02-18T11:48:47.310420+0000","last_clean":"2024-02-18T11:48:47.310420+0000","last_became_active":"2024-02-18T11:48:46.305252+0000","last_became_peered":"2024-02-18T11:48:46.305252+0000","last_unstale":"2024-02-18T11:48:47.310420+0000","last_undegraded":"2024-02-18T11:48:47.310420+0000","last_fullsized":"2024-02-18T11:48:47.310420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309454+0000","last_change":"2024-02-18T11:48:47.309454+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309454+0000","last_undegraded":"2024-02-18T11:48:47.309454+0000","last_fullsized":"2024-02-18T11:48:47.309454+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305829+0000","last_change":"2024-02-18T11:48:46.308595+0000","last_active":"2024-02-18T11:48:47.305829+0000","last_peered":"2024-02-18T11:48:47.305829+0000","last_clean":"2024-02-18T11:48:47.305829+0000","last_became_active":"2024-02-18T11:48:46.308361+0000","last_became_peered":"2024-02-18T11:48:46.308361+0000","last_unstale":"2024-02-18T11:48:47.305829+0000","last_undegraded":"2024-02-18T11:48:47.305829+0000","last_fullsized":"2024-02-18T11:48:47.305829+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305889+0000","last_change":"2024-02-18T11:48:47.305889+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305889+0000","last_undegraded":"2024-02-18T11:48:47.305889+0000","last_fullsized":"2024-02-18T11:48:47.305889+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308401+0000","last_change":"2024-02-18T11:48:46.300103+0000","last_active":"2024-02-18T11:48:47.308401+0000","last_peered":"2024-02-18T11:48:47.308401+0000","last_clean":"2024-02-18T11:48:47.308401+0000","last_became_active":"2024-02-18T11:48:46.299808+0000","last_became_peered":"2024-02-18T11:48:46.299808+0000","last_unstale":"2024-02-18T11:48:47.308401+0000","last_undegraded":"2024-02-18T11:48:47.308401+0000","last_fullsized":"2024-02-18T11:48:47.308401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320639+0000","last_change":"2024-02-18T11:48:47.320639+0000","last_active":"2024-02-18T11:48:47.320639+0000","last_peered":"2024-02-18T11:48:47.320639+0000","last_clean":"2024-02-18T11:48:47.320639+0000","last_became_active":"2024-02-18T11:48:47.320385+0000","last_became_peered":"2024-02-18T11:48:47.320385+0000","last_unstale":"2024-02-18T11:48:47.320639+0000","last_undegraded":"2024-02-18T11:48:47.320639+0000","last_fullsized":"2024-02-18T11:48:47.320639+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:57.144454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310023+0000","last_change":"2024-02-18T11:48:46.304701+0000","last_active":"2024-02-18T11:48:47.310023+0000","last_peered":"2024-02-18T11:48:47.310023+0000","last_clean":"2024-02-18T11:48:47.310023+0000","last_became_active":"2024-02-18T11:48:46.304450+0000","last_became_peered":"2024-02-18T11:48:46.304450+0000","last_unstale":"2024-02-18T11:48:47.310023+0000","last_undegraded":"2024-02-18T11:48:47.310023+0000","last_fullsized":"2024-02-18T11:48:47.310023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.317646+0000","last_change":"2024-02-18T11:48:47.317646+0000","last_active":"2024-02-18T11:48:47.317646+0000","last_peered":"2024-02-18T11:48:47.317646+0000","last_clean":"2024-02-18T11:48:47.317646+0000","last_became_active":"2024-02-18T11:48:47.317171+0000","last_became_peered":"2024-02-18T11:48:47.317171+0000","last_unstale":"2024-02-18T11:48:47.317646+0000","last_undegraded":"2024-02-18T11:48:47.317646+0000","last_fullsized":"2024-02-18T11:48:47.317646+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:14.550912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304637+0000","last_change":"2024-02-18T11:48:46.296450+0000","last_active":"2024-02-18T11:48:47.304637+0000","last_peered":"2024-02-18T11:48:47.304637+0000","last_clean":"2024-02-18T11:48:47.304637+0000","last_became_active":"2024-02-18T11:48:46.296159+0000","last_became_peered":"2024-02-18T11:48:46.296159+0000","last_unstale":"2024-02-18T11:48:47.304637+0000","last_undegraded":"2024-02-18T11:48:47.304637+0000","last_fullsized":"2024-02-18T11:48:47.304637+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.301870+0000","last_change":"2024-02-18T11:48:47.301870+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.301870+0000","last_undegraded":"2024-02-18T11:48:47.301870+0000","last_fullsized":"2024-02-18T11:48:47.301870+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303802+0000","last_change":"2024-02-18T11:48:46.306411+0000","last_active":"2024-02-18T11:48:47.303802+0000","last_peered":"2024-02-18T11:48:47.303802+0000","last_clean":"2024-02-18T11:48:47.303802+0000","last_became_active":"2024-02-18T11:48:46.304718+0000","last_became_peered":"2024-02-18T11:48:46.304718+0000","last_unstale":"2024-02-18T11:48:47.303802+0000","last_undegraded":"2024-02-18T11:48:47.303802+0000","last_fullsized":"2024-02-18T11:48:47.303802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311877+0000","last_change":"2024-02-18T11:48:47.311877+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311877+0000","last_undegraded":"2024-02-18T11:48:47.311877+0000","last_fullsized":"2024-02-18T11:48:47.311877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308363+0000","last_change":"2024-02-18T11:48:46.307561+0000","last_active":"2024-02-18T11:48:47.308363+0000","last_peered":"2024-02-18T11:48:47.308363+0000","last_clean":"2024-02-18T11:48:47.308363+0000","last_became_active":"2024-02-18T11:48:46.307253+0000","last_became_peered":"2024-02-18T11:48:46.307253+0000","last_unstale":"2024-02-18T11:48:47.308363+0000","last_undegraded":"2024-02-18T11:48:47.308363+0000","last_fullsized":"2024-02-18T11:48:47.308363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.316738+0000","last_change":"2024-02-18T11:48:47.316738+0000","last_active":"2024-02-18T11:48:47.316738+0000","last_peered":"2024-02-18T11:48:47.316738+0000","last_clean":"2024-02-18T11:48:47.316738+0000","last_became_active":"2024-02-18T11:48:47.316488+0000","last_became_peered":"2024-02-18T11:48:47.316488+0000","last_unstale":"2024-02-18T11:48:47.316738+0000","last_undegraded":"2024-02-18T11:48:47.316738+0000","last_fullsized":"2024-02-18T11:48:47.316738+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305690+0000","last_change":"2024-02-18T11:48:46.304153+0000","last_active":"2024-02-18T11:48:47.305690+0000","last_peered":"2024-02-18T11:48:47.305690+0000","last_clean":"2024-02-18T11:48:47.305690+0000","last_became_active":"2024-02-18T11:48:46.303687+0000","last_became_peered":"2024-02-18T11:48:46.303687+0000","last_unstale":"2024-02-18T11:48:47.305690+0000","last_undegraded":"2024-02-18T11:48:47.305690+0000","last_fullsized":"2024-02-18T11:48:47.305690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.307009+0000","last_change":"2024-02-18T11:48:47.307009+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.307009+0000","last_undegraded":"2024-02-18T11:48:47.307009+0000","last_fullsized":"2024-02-18T11:48:47.307009+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'5","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304472+0000","last_change":"2024-02-18T11:48:46.301330+0000","last_active":"2024-02-18T11:48:47.304472+0000","last_peered":"2024-02-18T11:48:47.304472+0000","last_clean":"2024-02-18T11:48:47.304472+0000","last_became_active":"2024-02-18T11:48:46.301062+0000","last_became_peered":"2024-02-18T11:48:46.301062+0000","last_unstale":"2024-02-18T11:48:47.304472+0000","last_undegraded":"2024-02-18T11:48:47.304472+0000","last_fullsized":"2024-02-18T11:48:47.304472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309147+0000","last_change":"2024-02-18T11:48:47.309147+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309147+0000","last_undegraded":"2024-02-18T11:48:47.309147+0000","last_fullsized":"2024-02-18T11:48:47.309147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.304790+0000","last_change":"2024-02-18T11:48:46.307286+0000","last_active":"2024-02-18T11:48:47.304790+0000","last_peered":"2024-02-18T11:48:47.304790+0000","last_clean":"2024-02-18T11:48:47.304790+0000","last_became_active":"2024-02-18T11:48:46.307002+0000","last_became_peered":"2024-02-18T11:48:46.307002+0000","last_unstale":"2024-02-18T11:48:47.304790+0000","last_undegraded":"2024-02-18T11:48:47.304790+0000","last_fullsized":"2024-02-18T11:48:47.304790+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:21.137008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323364+0000","last_change":"2024-02-18T11:48:47.323364+0000","last_active":"2024-02-18T11:48:47.323364+0000","last_peered":"2024-02-18T11:48:47.323364+0000","last_clean":"2024-02-18T11:48:47.323364+0000","last_became_active":"2024-02-18T11:48:47.318734+0000","last_became_peered":"2024-02-18T11:48:47.318734+0000","last_unstale":"2024-02-18T11:48:47.323364+0000","last_undegraded":"2024-02-18T11:48:47.323364+0000","last_fullsized":"2024-02-18T11:48:47.323364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:29:29.908143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306130+0000","last_change":"2024-02-18T11:48:46.305434+0000","last_active":"2024-02-18T11:48:47.306130+0000","last_peered":"2024-02-18T11:48:47.306130+0000","last_clean":"2024-02-18T11:48:47.306130+0000","last_became_active":"2024-02-18T11:48:46.305145+0000","last_became_peered":"2024-02-18T11:48:46.305145+0000","last_unstale":"2024-02-18T11:48:47.306130+0000","last_undegraded":"2024-02-18T11:48:47.306130+0000","last_fullsized":"2024-02-18T11:48:47.306130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:07.775308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308255+0000","last_change":"2024-02-18T11:48:47.308255+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308255+0000","last_undegraded":"2024-02-18T11:48:47.308255+0000","last_fullsized":"2024-02-18T11:48:47.308255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301908+0000","last_change":"2024-02-18T11:48:46.297664+0000","last_active":"2024-02-18T11:48:47.301908+0000","last_peered":"2024-02-18T11:48:47.301908+0000","last_clean":"2024-02-18T11:48:47.301908+0000","last_became_active":"2024-02-18T11:48:46.297202+0000","last_became_peered":"2024-02-18T11:48:46.297202+0000","last_unstale":"2024-02-18T11:48:47.301908+0000","last_undegraded":"2024-02-18T11:48:47.301908+0000","last_fullsized":"2024-02-18T11:48:47.301908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.318448+0000","last_change":"2024-02-18T11:48:47.318448+0000","last_active":"2024-02-18T11:48:47.318448+0000","last_peered":"2024-02-18T11:48:47.318448+0000","last_clean":"2024-02-18T11:48:47.318448+0000","last_became_active":"2024-02-18T11:48:47.318190+0000","last_became_peered":"2024-02-18T11:48:47.318190+0000","last_unstale":"2024-02-18T11:48:47.318448+0000","last_undegraded":"2024-02-18T11:48:47.318448+0000","last_fullsized":"2024-02-18T11:48:47.318448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:15:45.332480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'4","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306501+0000","last_change":"2024-02-18T11:48:46.300667+0000","last_active":"2024-02-18T11:48:47.306501+0000","last_peered":"2024-02-18T11:48:47.306501+0000","last_clean":"2024-02-18T11:48:47.306501+0000","last_became_active":"2024-02-18T11:48:46.300407+0000","last_became_peered":"2024-02-18T11:48:46.300407+0000","last_unstale":"2024-02-18T11:48:47.306501+0000","last_undegraded":"2024-02-18T11:48:47.306501+0000","last_fullsized":"2024-02-18T11:48:47.306501+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.323994+0000","last_change":"2024-02-18T11:48:47.309612+0000","last_active":"2024-02-18T11:48:47.323994+0000","last_peered":"2024-02-18T11:48:47.323994+0000","last_clean":"2024-02-18T11:48:47.323994+0000","last_became_active":"2024-02-18T11:48:47.309382+0000","last_became_peered":"2024-02-18T11:48:47.309382+0000","last_unstale":"2024-02-18T11:48:47.323994+0000","last_undegraded":"2024-02-18T11:48:47.323994+0000","last_fullsized":"2024-02-18T11:48:47.323994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311657+0000","last_change":"2024-02-18T11:48:46.307646+0000","last_active":"2024-02-18T11:48:47.311657+0000","last_peered":"2024-02-18T11:48:47.311657+0000","last_clean":"2024-02-18T11:48:47.311657+0000","last_became_active":"2024-02-18T11:48:46.307345+0000","last_became_peered":"2024-02-18T11:48:46.307345+0000","last_unstale":"2024-02-18T11:48:47.311657+0000","last_undegraded":"2024-02-18T11:48:47.311657+0000","last_fullsized":"2024-02-18T11:48:47.311657+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:50.554971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.304896+0000","last_change":"2024-02-18T11:48:47.304896+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.304896+0000","last_undegraded":"2024-02-18T11:48:47.304896+0000","last_fullsized":"2024-02-18T11:48:47.304896+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306767+0000","last_change":"2024-02-18T11:48:46.306743+0000","last_active":"2024-02-18T11:48:47.306767+0000","last_peered":"2024-02-18T11:48:47.306767+0000","last_clean":"2024-02-18T11:48:47.306767+0000","last_became_active":"2024-02-18T11:48:46.306447+0000","last_became_peered":"2024-02-18T11:48:46.306447+0000","last_unstale":"2024-02-18T11:48:47.306767+0000","last_undegraded":"2024-02-18T11:48:47.306767+0000","last_fullsized":"2024-02-18T11:48:47.306767+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.321271+0000","last_change":"2024-02-18T11:48:47.321271+0000","last_active":"2024-02-18T11:48:47.321271+0000","last_peered":"2024-02-18T11:48:47.321271+0000","last_clean":"2024-02-18T11:48:47.321271+0000","last_became_active":"2024-02-18T11:48:47.321021+0000","last_became_peered":"2024-02-18T11:48:47.321021+0000","last_unstale":"2024-02-18T11:48:47.321271+0000","last_undegraded":"2024-02-18T11:48:47.321271+0000","last_fullsized":"2024-02-18T11:48:47.321271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301977+0000","last_change":"2024-02-18T11:48:46.296023+0000","last_active":"2024-02-18T11:48:47.301977+0000","last_peered":"2024-02-18T11:48:47.301977+0000","last_clean":"2024-02-18T11:48:47.301977+0000","last_became_active":"2024-02-18T11:48:46.295715+0000","last_became_peered":"2024-02-18T11:48:46.295715+0000","last_unstale":"2024-02-18T11:48:47.301977+0000","last_undegraded":"2024-02-18T11:48:47.301977+0000","last_fullsized":"2024-02-18T11:48:47.301977+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310131+0000","last_change":"2024-02-18T11:48:47.310131+0000","last_active":"2024-02-18T11:48:47.310131+0000","last_peered":"2024-02-18T11:48:47.310131+0000","last_clean":"2024-02-18T11:48:47.310131+0000","last_became_active":"2024-02-18T11:48:47.309877+0000","last_became_peered":"2024-02-18T11:48:47.309877+0000","last_unstale":"2024-02-18T11:48:47.310131+0000","last_undegraded":"2024-02-18T11:48:47.310131+0000","last_fullsized":"2024-02-18T11:48:47.310131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:49.971716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305646+0000","last_change":"2024-02-18T11:48:46.297720+0000","last_active":"2024-02-18T11:48:47.305646+0000","last_peered":"2024-02-18T11:48:47.305646+0000","last_clean":"2024-02-18T11:48:47.305646+0000","last_became_active":"2024-02-18T11:48:46.297462+0000","last_became_peered":"2024-02-18T11:48:46.297462+0000","last_unstale":"2024-02-18T11:48:47.305646+0000","last_undegraded":"2024-02-18T11:48:47.305646+0000","last_fullsized":"2024-02-18T11:48:47.305646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306864+0000","last_change":"2024-02-18T11:48:47.306864+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306864+0000","last_undegraded":"2024-02-18T11:48:47.306864+0000","last_fullsized":"2024-02-18T11:48:47.306864+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301800+0000","last_change":"2024-02-18T11:48:46.303136+0000","last_active":"2024-02-18T11:48:47.301800+0000","last_peered":"2024-02-18T11:48:47.301800+0000","last_clean":"2024-02-18T11:48:47.301800+0000","last_became_active":"2024-02-18T11:48:46.302861+0000","last_became_peered":"2024-02-18T11:48:46.302861+0000","last_unstale":"2024-02-18T11:48:47.301800+0000","last_undegraded":"2024-02-18T11:48:47.301800+0000","last_fullsized":"2024-02-18T11:48:47.301800+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.306944+0000","last_change":"2024-02-18T11:48:47.306944+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.306944+0000","last_undegraded":"2024-02-18T11:48:47.306944+0000","last_fullsized":"2024-02-18T11:48:47.306944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.300950+0000","last_change":"2024-02-18T11:48:46.297761+0000","last_active":"2024-02-18T11:48:47.300950+0000","last_peered":"2024-02-18T11:48:47.300950+0000","last_clean":"2024-02-18T11:48:47.300950+0000","last_became_active":"2024-02-18T11:48:46.297500+0000","last_became_peered":"2024-02-18T11:48:46.297500+0000","last_unstale":"2024-02-18T11:48:47.300950+0000","last_undegraded":"2024-02-18T11:48:47.300950+0000","last_fullsized":"2024-02-18T11:48:47.300950+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:33:10.032538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.310081+0000","last_change":"2024-02-18T11:48:47.310081+0000","last_active":"2024-02-18T11:48:47.310081+0000","last_peered":"2024-02-18T11:48:47.310081+0000","last_clean":"2024-02-18T11:48:47.310081+0000","last_became_active":"2024-02-18T11:48:47.309813+0000","last_became_peered":"2024-02-18T11:48:47.309813+0000","last_unstale":"2024-02-18T11:48:47.310081+0000","last_undegraded":"2024-02-18T11:48:47.310081+0000","last_fullsized":"2024-02-18T11:48:47.310081+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.306548+0000","last_change":"2024-02-18T11:48:46.298569+0000","last_active":"2024-02-18T11:48:47.306548+0000","last_peered":"2024-02-18T11:48:47.306548+0000","last_clean":"2024-02-18T11:48:47.306548+0000","last_became_active":"2024-02-18T11:48:46.297865+0000","last_became_peered":"2024-02-18T11:48:46.297865+0000","last_unstale":"2024-02-18T11:48:47.306548+0000","last_undegraded":"2024-02-18T11:48:47.306548+0000","last_fullsized":"2024-02-18T11:48:47.306548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:04.067605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.312852+0000","last_change":"2024-02-18T11:48:47.312852+0000","last_active":"2024-02-18T11:48:47.312852+0000","last_peered":"2024-02-18T11:48:47.312852+0000","last_clean":"2024-02-18T11:48:47.312852+0000","last_became_active":"2024-02-18T11:48:47.312584+0000","last_became_peered":"2024-02-18T11:48:47.312584+0000","last_unstale":"2024-02-18T11:48:47.312852+0000","last_undegraded":"2024-02-18T11:48:47.312852+0000","last_fullsized":"2024-02-18T11:48:47.312852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:51.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301208+0000","last_change":"2024-02-18T11:48:46.302429+0000","last_active":"2024-02-18T11:48:47.301208+0000","last_peered":"2024-02-18T11:48:47.301208+0000","last_clean":"2024-02-18T11:48:47.301208+0000","last_became_active":"2024-02-18T11:48:46.302138+0000","last_became_peered":"2024-02-18T11:48:46.302138+0000","last_unstale":"2024-02-18T11:48:47.301208+0000","last_undegraded":"2024-02-18T11:48:47.301208+0000","last_fullsized":"2024-02-18T11:48:47.301208+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:54.091351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.305364+0000","last_change":"2024-02-18T11:48:47.305364+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.305364+0000","last_undegraded":"2024-02-18T11:48:47.305364+0000","last_fullsized":"2024-02-18T11:48:47.305364+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.311359+0000","last_change":"2024-02-18T11:48:46.308271+0000","last_active":"2024-02-18T11:48:47.311359+0000","last_peered":"2024-02-18T11:48:47.311359+0000","last_clean":"2024-02-18T11:48:47.311359+0000","last_became_active":"2024-02-18T11:48:46.307981+0000","last_became_peered":"2024-02-18T11:48:46.307981+0000","last_unstale":"2024-02-18T11:48:47.311359+0000","last_undegraded":"2024-02-18T11:48:47.311359+0000","last_fullsized":"2024-02-18T11:48:47.311359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:32.990774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311673+0000","last_change":"2024-02-18T11:48:47.311673+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311673+0000","last_undegraded":"2024-02-18T11:48:47.311673+0000","last_fullsized":"2024-02-18T11:48:47.311673+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.301714+0000","last_change":"2024-02-18T11:48:46.305551+0000","last_active":"2024-02-18T11:48:47.301714+0000","last_peered":"2024-02-18T11:48:47.301714+0000","last_clean":"2024-02-18T11:48:47.301714+0000","last_became_active":"2024-02-18T11:48:46.305246+0000","last_became_peered":"2024-02-18T11:48:46.305246+0000","last_unstale":"2024-02-18T11:48:47.301714+0000","last_undegraded":"2024-02-18T11:48:47.301714+0000","last_fullsized":"2024-02-18T11:48:47.301714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.319083+0000","last_change":"2024-02-18T11:48:47.319083+0000","last_active":"2024-02-18T11:48:47.319083+0000","last_peered":"2024-02-18T11:48:47.319083+0000","last_clean":"2024-02-18T11:48:47.319083+0000","last_became_active":"2024-02-18T11:48:47.318762+0000","last_became_peered":"2024-02-18T11:48:47.318762+0000","last_unstale":"2024-02-18T11:48:47.319083+0000","last_undegraded":"2024-02-18T11:48:47.319083+0000","last_fullsized":"2024-02-18T11:48:47.319083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:55:35.888377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.305354+0000","last_change":"2024-02-18T11:48:46.305494+0000","last_active":"2024-02-18T11:48:47.305354+0000","last_peered":"2024-02-18T11:48:47.305354+0000","last_clean":"2024-02-18T11:48:47.305354+0000","last_became_active":"2024-02-18T11:48:46.305222+0000","last_became_peered":"2024-02-18T11:48:46.305222+0000","last_unstale":"2024-02-18T11:48:47.305354+0000","last_undegraded":"2024-02-18T11:48:47.305354+0000","last_fullsized":"2024-02-18T11:48:47.305354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.308833+0000","last_change":"2024-02-18T11:48:47.308833+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.308833+0000","last_undegraded":"2024-02-18T11:48:47.308833+0000","last_fullsized":"2024-02-18T11:48:47.308833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.308558+0000","last_change":"2024-02-18T11:48:46.303722+0000","last_active":"2024-02-18T11:48:47.308558+0000","last_peered":"2024-02-18T11:48:47.308558+0000","last_clean":"2024-02-18T11:48:47.308558+0000","last_became_active":"2024-02-18T11:48:46.303460+0000","last_became_peered":"2024-02-18T11:48:46.303460+0000","last_unstale":"2024-02-18T11:48:47.308558+0000","last_undegraded":"2024-02-18T11:48:47.308558+0000","last_fullsized":"2024-02-18T11:48:47.308558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.309759+0000","last_change":"2024-02-18T11:48:47.309759+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.309759+0000","last_undegraded":"2024-02-18T11:48:47.309759+0000","last_fullsized":"2024-02-18T11:48:47.309759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.302363+0000","last_change":"2024-02-18T11:48:46.299505+0000","last_active":"2024-02-18T11:48:47.302363+0000","last_peered":"2024-02-18T11:48:47.302363+0000","last_clean":"2024-02-18T11:48:47.302363+0000","last_became_active":"2024-02-18T11:48:46.299230+0000","last_became_peered":"2024-02-18T11:48:46.299230+0000","last_unstale":"2024-02-18T11:48:47.302363+0000","last_undegraded":"2024-02-18T11:48:47.302363+0000","last_fullsized":"2024-02-18T11:48:47.302363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.320820+0000","last_change":"2024-02-18T11:48:47.315114+0000","last_active":"2024-02-18T11:48:47.320820+0000","last_peered":"2024-02-18T11:48:47.320820+0000","last_clean":"2024-02-18T11:48:47.320820+0000","last_became_active":"2024-02-18T11:48:47.313064+0000","last_became_peered":"2024-02-18T11:48:47.313064+0000","last_unstale":"2024-02-18T11:48:47.320820+0000","last_undegraded":"2024-02-18T11:48:47.320820+0000","last_fullsized":"2024-02-18T11:48:47.320820+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:22.773038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307959+0000","last_change":"2024-02-18T11:48:46.306424+0000","last_active":"2024-02-18T11:48:47.307959+0000","last_peered":"2024-02-18T11:48:47.307959+0000","last_clean":"2024-02-18T11:48:47.307959+0000","last_became_active":"2024-02-18T11:48:46.305938+0000","last_became_peered":"2024-02-18T11:48:46.305938+0000","last_unstale":"2024-02-18T11:48:47.307959+0000","last_undegraded":"2024-02-18T11:48:47.307959+0000","last_fullsized":"2024-02-18T11:48:47.307959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:48.967965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":8,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.314001+0000","last_change":"2024-02-18T11:48:47.314001+0000","last_active":"2024-02-18T11:48:47.314001+0000","last_peered":"2024-02-18T11:48:47.314001+0000","last_clean":"2024-02-18T11:48:47.314001+0000","last_became_active":"2024-02-18T11:48:47.313680+0000","last_became_peered":"2024-02-18T11:48:47.313680+0000","last_unstale":"2024-02-18T11:48:47.314001+0000","last_undegraded":"2024-02-18T11:48:47.314001+0000","last_fullsized":"2024-02-18T11:48:47.314001+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.303228+0000","last_change":"2024-02-18T11:48:46.306107+0000","last_active":"2024-02-18T11:48:47.303228+0000","last_peered":"2024-02-18T11:48:47.303228+0000","last_clean":"2024-02-18T11:48:47.303228+0000","last_became_active":"2024-02-18T11:48:46.305749+0000","last_became_peered":"2024-02-18T11:48:46.305749+0000","last_unstale":"2024-02-18T11:48:47.303228+0000","last_undegraded":"2024-02-18T11:48:47.303228+0000","last_fullsized":"2024-02-18T11:48:47.303228+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-02-18T11:48:47.311076+0000","last_change":"2024-02-18T11:48:47.311076+0000","last_active":"2024-02-18T11:48:46.281573+0000","last_peered":"2024-02-18T11:48:46.281573+0000","last_clean":"2024-02-18T11:48:46.281573+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-02-18T11:48:47.311076+0000","last_undegraded":"2024-02-18T11:48:47.311076+0000","last_fullsized":"2024-02-18T11:48:47.311076+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307802+0000","last_change":"2024-02-18T11:48:46.303447+0000","last_active":"2024-02-18T11:48:47.307802+0000","last_peered":"2024-02-18T11:48:47.307802+0000","last_clean":"2024-02-18T11:48:47.307802+0000","last_became_active":"2024-02-18T11:48:46.303126+0000","last_became_peered":"2024-02-18T11:48:46.303126+0000","last_unstale":"2024-02-18T11:48:47.307802+0000","last_undegraded":"2024-02-18T11:48:47.307802+0000","last_fullsized":"2024-02-18T11:48:47.307802+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:44.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":201,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-02-18T11:48:47.307052+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:48:47.307052+0000","last_peered":"2024-02-18T11:48:47.307052+0000","last_clean":"2024-02-18T11:48:47.307052+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:48:47.307052+0000","last_undegraded":"2024-02-18T11:48:47.307052+0000","last_fullsized":"2024-02-18T11:48:47.307052+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1288,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21,"ondisk_log_size":21,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":183498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":413696,"data_stored":176331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":766699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:54.641 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-18T11:48:54.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.760+0000 7f3dd2484640 1 Processor -- start 2024-02-18T11:48:54.753 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.760+0000 7f3dd2484640 1 -- start start 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dd2484640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3dcc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dd2484640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc0b02e0 con 0x7f3dcc0cd1f0 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dd2484640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc0af6d0 con 0x7f3dcc06ea20 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dd2484640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0af9b0 con 0x7f3dcc15a860 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcb7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3dcc0cd1f0 0x7f3dcc06e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34772/0 (socket says 172.21.15.116:34772) 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/249082352 learned_addr learned my addr 172.21.15.116:0/249082352 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcbfff640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3dcc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300634192 0 0) 0x7f3dcc0b02e0 con 0x7f3dcc0cd1f0 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db4003520 con 0x7f3dcc0cd1f0 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599564104 0 0) 0x7f3dcc0af6d0 con 0x7f3dcc06ea20 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dcc0b02e0 con 0x7f3dcc06ea20 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1455537319 0 0) 0x7f3db4003520 con 0x7f3dcc0cd1f0 2024-02-18T11:48:54.758 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 msgr2=0x7f3dcc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3dcc0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 >> v1:172.21.15.143:6789/0 conn(0x7f3dcc06ea20 legacy=0x7f3dcc0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0afc90 con 0x7f3dcc0cd1f0 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f3dbc001670 con 0x7f3dcc0cd1f0 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 >> v1:172.21.15.116:6789/0 conn(0x7f3dcc0cd1f0 legacy=0x7f3dcc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db4003b30 0x7f3db4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3db40047c0 0x7f3db4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3db4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0af9b0 con 0x7f3db40047c0 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db400ba30 con 0x7f3db4003b30 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.761+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db400bbc0 con 0x7f3dcc15a860 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3db40047c0 0x7f3db4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcbfff640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db4003b30 0x7f3db4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.759 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dd0a21640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3db4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 msgr2=0x7f3db4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3db4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db4003b30 msgr2=0x7f3db4003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db4003b30 0x7f3db4003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/249082352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4003520 con 0x7f3db40047c0 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.762+0000 7f3dd0a21640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3dcc15a860 0x7f3db4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3db40047c0 0x7f3db4004c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007fd0 tx=0x7f3db40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f9f150ccbfaec0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc005ad0 con 0x7f3db40047c0 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc005c50 con 0x7f3db40047c0 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dcaffd640 1 -- 172.21.15.116:0/249082352 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc02c7c0 con 0x7f3db40047c0 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd2484640 1 -- 172.21.15.116:0/249082352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3db40047c0 msgr2=0x7f3db4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd2484640 1 --2- 172.21.15.116:0/249082352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3db40047c0 0x7f3db4004c40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc007fd0 tx=0x7f3db40057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.760 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd0a21640 1 -- 172.21.15.116:0/249082352 reap_dead start 2024-02-18T11:48:54.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd2484640 1 -- 172.21.15.116:0/249082352 shutdown_connections 2024-02-18T11:48:54.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd2484640 1 -- 172.21.15.116:0/249082352 >> 172.21.15.116:0/249082352 conn(0x7f3dcc066f20 msgr2=0x7f3dcc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:54.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd2484640 1 -- 172.21.15.116:0/249082352 shutdown_connections 2024-02-18T11:48:54.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.763+0000 7f3dd2484640 1 -- 172.21.15.116:0/249082352 wait complete. 2024-02-18T11:48:54.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.764+0000 7f3dd2484640 1 Processor -- start 2024-02-18T11:48:54.761 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 -- start start 2024-02-18T11:48:54.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3db40047c0 0x7f3dcc1411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 0x7f3dcc1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.763 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dcc157f10 0x7f3dcc151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc156930 con 0x7f3dcc0cc160 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc156c10 con 0x7f3dcc157f10 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dd2484640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc156ef0 con 0x7f3db40047c0 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 0x7f3dcc1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dcb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 0x7f3dcc1579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42028/0 (socket says 172.21.15.116:42028) 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.765+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/124618628 learned_addr learned my addr 172.21.15.116:0/124618628 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dd0a21640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dcc157f10 0x7f3dcc151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3db40047c0 msgr2=0x7f3dcc1411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3db40047c0 0x7f3dcc1411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dcc157f10 msgr2=0x7f3dcc151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dcc157f10 0x7f3dcc151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dcb7fe640 1 -- 172.21.15.116:0/124618628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc1571d0 con 0x7f3dcc0cc160 2024-02-18T11:48:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dd0a21640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dcc157f10 0x7f3dcc151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dcb7fe640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 0x7f3dcc1579d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0018b0 tx=0x7f3dbc0019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc03c040 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc02cbd0 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc1574b0 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc007070 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.766+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc0ade50 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.767+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc157640 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.767+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3dbc032070 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.768+0000 7f3dc8ff9640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3db806bc70 0x7f3db806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.768+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3dbc0abe10 con 0x7f3dcc0cc160 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.768+0000 7f3dcbfff640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3db806bc70 0x7f3db806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.768+0000 7f3dcbfff640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3db806bc70 0x7f3db806e130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00099e0 tx=0x7f3dc0006c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:54.766 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.772+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3dbc0765a0 con 0x7f3dcc0cc160 2024-02-18T11:48:54.933 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.940+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 --> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dcc156d80 con 0x7f3db806bc70 2024-02-18T11:48:54.939 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.946+0000 7f3dc8ff9640 1 -- 172.21.15.116:0/124618628 <== mgr.4101 v2:172.21.15.116:6832/4274496429 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7f3dcc156d80 con 0x7f3db806bc70 2024-02-18T11:48:54.942 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:54.943 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2024-02-18T11:48:54.944 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3db806bc70 msgr2=0x7f3db806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3db806bc70 0x7f3db806e130 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00099e0 tx=0x7f3dc0006c30 comp rx=0 tx=0).stop 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 msgr2=0x7f3dcc1579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 0x7f3dcc1579d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0018b0 tx=0x7f3dbc0019f0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 shutdown_connections 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f3db806bc70 0x7f3db806e130 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dcc157f10 0x7f3dcc151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3dcc0cc160 0x7f3dcc1579d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 --2- 172.21.15.116:0/124618628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3db40047c0 0x7f3dcc1411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 >> 172.21.15.116:0/124618628 conn(0x7f3dcc066f20 msgr2=0x7f3dcc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 shutdown_connections 2024-02-18T11:48:54.945 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:54.949+0000 7f3dd2484640 1 -- 172.21.15.116:0/124618628 wait complete. 2024-02-18T11:48:54.953 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-02-18T11:48:54.307132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220844,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753876,"statfs":{"total":773094113280,"available":772867969024,"internally_reserved":0,"allocated":4759552,"data_stored":2707076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001527"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850852+0000","last_change":"2024-02-18T11:48:47.319567+0000","last_active":"2024-02-18T11:48:48.850852+0000","last_peered":"2024-02-18T11:48:48.850852+0000","last_clean":"2024-02-18T11:48:48.850852+0000","last_became_active":"2024-02-18T11:48:47.318466+0000","last_became_peered":"2024-02-18T11:48:47.318466+0000","last_unstale":"2024-02-18T11:48:48.850852+0000","last_undegraded":"2024-02-18T11:48:48.850852+0000","last_fullsized":"2024-02-18T11:48:48.850852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:11:58.023116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850648+0000","last_change":"2024-02-18T11:48:47.317654+0000","last_active":"2024-02-18T11:48:48.850648+0000","last_peered":"2024-02-18T11:48:48.850648+0000","last_clean":"2024-02-18T11:48:48.850648+0000","last_became_active":"2024-02-18T11:48:47.316735+0000","last_became_peered":"2024-02-18T11:48:47.316735+0000","last_unstale":"2024-02-18T11:48:48.850648+0000","last_undegraded":"2024-02-18T11:48:48.850648+0000","last_fullsized":"2024-02-18T11:48:48.850648+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:20:25.909108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849899+0000","last_change":"2024-02-18T11:48:47.317671+0000","last_active":"2024-02-18T11:48:48.849899+0000","last_peered":"2024-02-18T11:48:48.849899+0000","last_clean":"2024-02-18T11:48:48.849899+0000","last_became_active":"2024-02-18T11:48:47.317384+0000","last_became_peered":"2024-02-18T11:48:47.317384+0000","last_unstale":"2024-02-18T11:48:48.849899+0000","last_undegraded":"2024-02-18T11:48:48.849899+0000","last_fullsized":"2024-02-18T11:48:48.849899+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:35:02.679276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315203+0000","last_change":"2024-02-18T11:48:47.326104+0000","last_active":"2024-02-18T11:48:48.315203+0000","last_peered":"2024-02-18T11:48:48.315203+0000","last_clean":"2024-02-18T11:48:48.315203+0000","last_became_active":"2024-02-18T11:48:47.325824+0000","last_became_peered":"2024-02-18T11:48:47.325824+0000","last_unstale":"2024-02-18T11:48:48.315203+0000","last_undegraded":"2024-02-18T11:48:48.315203+0000","last_fullsized":"2024-02-18T11:48:48.315203+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:37.307682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315183+0000","last_change":"2024-02-18T11:48:47.318045+0000","last_active":"2024-02-18T11:48:48.315183+0000","last_peered":"2024-02-18T11:48:48.315183+0000","last_clean":"2024-02-18T11:48:48.315183+0000","last_became_active":"2024-02-18T11:48:47.317778+0000","last_became_peered":"2024-02-18T11:48:47.317778+0000","last_unstale":"2024-02-18T11:48:48.315183+0000","last_undegraded":"2024-02-18T11:48:48.315183+0000","last_fullsized":"2024-02-18T11:48:48.315183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849785+0000","last_change":"2024-02-18T11:48:47.315189+0000","last_active":"2024-02-18T11:48:48.849785+0000","last_peered":"2024-02-18T11:48:48.849785+0000","last_clean":"2024-02-18T11:48:48.849785+0000","last_became_active":"2024-02-18T11:48:47.314889+0000","last_became_peered":"2024-02-18T11:48:47.314889+0000","last_unstale":"2024-02-18T11:48:48.849785+0000","last_undegraded":"2024-02-18T11:48:48.849785+0000","last_fullsized":"2024-02-18T11:48:48.849785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:51.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850407+0000","last_change":"2024-02-18T11:48:47.329825+0000","last_active":"2024-02-18T11:48:48.850407+0000","last_peered":"2024-02-18T11:48:48.850407+0000","last_clean":"2024-02-18T11:48:48.850407+0000","last_became_active":"2024-02-18T11:48:47.329233+0000","last_became_peered":"2024-02-18T11:48:47.329233+0000","last_unstale":"2024-02-18T11:48:48.850407+0000","last_undegraded":"2024-02-18T11:48:48.850407+0000","last_fullsized":"2024-02-18T11:48:48.850407+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:52:59.755152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850350+0000","last_change":"2024-02-18T11:48:47.319168+0000","last_active":"2024-02-18T11:48:48.850350+0000","last_peered":"2024-02-18T11:48:48.850350+0000","last_clean":"2024-02-18T11:48:48.850350+0000","last_became_active":"2024-02-18T11:48:47.318918+0000","last_became_peered":"2024-02-18T11:48:47.318918+0000","last_unstale":"2024-02-18T11:48:48.850350+0000","last_undegraded":"2024-02-18T11:48:48.850350+0000","last_fullsized":"2024-02-18T11:48:48.850350+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:55:53.789541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850105+0000","last_change":"2024-02-18T11:48:47.328694+0000","last_active":"2024-02-18T11:48:48.850105+0000","last_peered":"2024-02-18T11:48:48.850105+0000","last_clean":"2024-02-18T11:48:48.850105+0000","last_became_active":"2024-02-18T11:48:47.328393+0000","last_became_peered":"2024-02-18T11:48:47.328393+0000","last_unstale":"2024-02-18T11:48:48.850105+0000","last_undegraded":"2024-02-18T11:48:48.850105+0000","last_fullsized":"2024-02-18T11:48:48.850105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:55:53.789541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850752+0000","last_change":"2024-02-18T11:48:47.318450+0000","last_active":"2024-02-18T11:48:48.850752+0000","last_peered":"2024-02-18T11:48:48.850752+0000","last_clean":"2024-02-18T11:48:48.850752+0000","last_became_active":"2024-02-18T11:48:47.317587+0000","last_became_peered":"2024-02-18T11:48:47.317587+0000","last_unstale":"2024-02-18T11:48:48.850752+0000","last_undegraded":"2024-02-18T11:48:48.850752+0000","last_fullsized":"2024-02-18T11:48:48.850752+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:13:45.002912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851013+0000","last_change":"2024-02-18T11:48:47.330526+0000","last_active":"2024-02-18T11:48:48.851013+0000","last_peered":"2024-02-18T11:48:48.851013+0000","last_clean":"2024-02-18T11:48:48.851013+0000","last_became_active":"2024-02-18T11:48:47.329155+0000","last_became_peered":"2024-02-18T11:48:47.329155+0000","last_unstale":"2024-02-18T11:48:48.851013+0000","last_undegraded":"2024-02-18T11:48:48.851013+0000","last_fullsized":"2024-02-18T11:48:48.851013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:07.500397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849861+0000","last_change":"2024-02-18T11:48:47.321118+0000","last_active":"2024-02-18T11:48:48.849861+0000","last_peered":"2024-02-18T11:48:48.849861+0000","last_clean":"2024-02-18T11:48:48.849861+0000","last_became_active":"2024-02-18T11:48:47.320856+0000","last_became_peered":"2024-02-18T11:48:47.320856+0000","last_unstale":"2024-02-18T11:48:48.849861+0000","last_undegraded":"2024-02-18T11:48:48.849861+0000","last_fullsized":"2024-02-18T11:48:48.849861+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:08:26.674078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850306+0000","last_change":"2024-02-18T11:48:47.317758+0000","last_active":"2024-02-18T11:48:48.850306+0000","last_peered":"2024-02-18T11:48:48.850306+0000","last_clean":"2024-02-18T11:48:48.850306+0000","last_became_active":"2024-02-18T11:48:47.317483+0000","last_became_peered":"2024-02-18T11:48:47.317483+0000","last_unstale":"2024-02-18T11:48:48.850306+0000","last_undegraded":"2024-02-18T11:48:48.850306+0000","last_fullsized":"2024-02-18T11:48:48.850306+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:54.435871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850195+0000","last_change":"2024-02-18T11:48:47.314842+0000","last_active":"2024-02-18T11:48:48.850195+0000","last_peered":"2024-02-18T11:48:48.850195+0000","last_clean":"2024-02-18T11:48:48.850195+0000","last_became_active":"2024-02-18T11:48:47.312762+0000","last_became_peered":"2024-02-18T11:48:47.312762+0000","last_unstale":"2024-02-18T11:48:48.850195+0000","last_undegraded":"2024-02-18T11:48:48.850195+0000","last_fullsized":"2024-02-18T11:48:48.850195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850604+0000","last_change":"2024-02-18T11:48:47.324590+0000","last_active":"2024-02-18T11:48:48.850604+0000","last_peered":"2024-02-18T11:48:48.850604+0000","last_clean":"2024-02-18T11:48:48.850604+0000","last_became_active":"2024-02-18T11:48:47.322687+0000","last_became_peered":"2024-02-18T11:48:47.322687+0000","last_unstale":"2024-02-18T11:48:48.850604+0000","last_undegraded":"2024-02-18T11:48:48.850604+0000","last_fullsized":"2024-02-18T11:48:48.850604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:51:54.332974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315866+0000","last_change":"2024-02-18T11:48:47.320840+0000","last_active":"2024-02-18T11:48:48.315866+0000","last_peered":"2024-02-18T11:48:48.315866+0000","last_clean":"2024-02-18T11:48:48.315866+0000","last_became_active":"2024-02-18T11:48:47.320257+0000","last_became_peered":"2024-02-18T11:48:47.320257+0000","last_unstale":"2024-02-18T11:48:48.315866+0000","last_undegraded":"2024-02-18T11:48:48.315866+0000","last_fullsized":"2024-02-18T11:48:48.315866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:35:02.679276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.317185+0000","last_change":"2024-02-18T11:48:47.330131+0000","last_active":"2024-02-18T11:48:48.317185+0000","last_peered":"2024-02-18T11:48:48.317185+0000","last_clean":"2024-02-18T11:48:48.317185+0000","last_became_active":"2024-02-18T11:48:47.329758+0000","last_became_peered":"2024-02-18T11:48:47.329758+0000","last_unstale":"2024-02-18T11:48:48.317185+0000","last_undegraded":"2024-02-18T11:48:48.317185+0000","last_fullsized":"2024-02-18T11:48:48.317185+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:15:38.182081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316330+0000","last_change":"2024-02-18T11:48:47.320419+0000","last_active":"2024-02-18T11:48:48.316330+0000","last_peered":"2024-02-18T11:48:48.316330+0000","last_clean":"2024-02-18T11:48:48.316330+0000","last_became_active":"2024-02-18T11:48:47.320054+0000","last_became_peered":"2024-02-18T11:48:47.320054+0000","last_unstale":"2024-02-18T11:48:48.316330+0000","last_undegraded":"2024-02-18T11:48:48.316330+0000","last_fullsized":"2024-02-18T11:48:48.316330+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:23.167273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315264+0000","last_change":"2024-02-18T11:48:47.318053+0000","last_active":"2024-02-18T11:48:48.315264+0000","last_peered":"2024-02-18T11:48:48.315264+0000","last_clean":"2024-02-18T11:48:48.315264+0000","last_became_active":"2024-02-18T11:48:47.317802+0000","last_became_peered":"2024-02-18T11:48:47.317802+0000","last_unstale":"2024-02-18T11:48:48.315264+0000","last_undegraded":"2024-02-18T11:48:48.315264+0000","last_fullsized":"2024-02-18T11:48:48.315264+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:45.431636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316257+0000","last_change":"2024-02-18T11:48:47.322363+0000","last_active":"2024-02-18T11:48:48.316257+0000","last_peered":"2024-02-18T11:48:48.316257+0000","last_clean":"2024-02-18T11:48:48.316257+0000","last_became_active":"2024-02-18T11:48:47.322113+0000","last_became_peered":"2024-02-18T11:48:47.322113+0000","last_unstale":"2024-02-18T11:48:48.316257+0000","last_undegraded":"2024-02-18T11:48:48.316257+0000","last_fullsized":"2024-02-18T11:48:48.316257+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:34.883282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850133+0000","last_change":"2024-02-18T11:48:47.327319+0000","last_active":"2024-02-18T11:48:48.850133+0000","last_peered":"2024-02-18T11:48:48.850133+0000","last_clean":"2024-02-18T11:48:48.850133+0000","last_became_active":"2024-02-18T11:48:47.326950+0000","last_became_peered":"2024-02-18T11:48:47.326950+0000","last_unstale":"2024-02-18T11:48:48.850133+0000","last_undegraded":"2024-02-18T11:48:48.850133+0000","last_fullsized":"2024-02-18T11:48:48.850133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:39:17.095580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851231+0000","last_change":"2024-02-18T11:48:47.324441+0000","last_active":"2024-02-18T11:48:48.851231+0000","last_peered":"2024-02-18T11:48:48.851231+0000","last_clean":"2024-02-18T11:48:48.851231+0000","last_became_active":"2024-02-18T11:48:47.324121+0000","last_became_peered":"2024-02-18T11:48:47.324121+0000","last_unstale":"2024-02-18T11:48:48.851231+0000","last_undegraded":"2024-02-18T11:48:48.851231+0000","last_fullsized":"2024-02-18T11:48:48.851231+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316471+0000","last_change":"2024-02-18T11:48:47.323370+0000","last_active":"2024-02-18T11:48:48.316471+0000","last_peered":"2024-02-18T11:48:48.316471+0000","last_clean":"2024-02-18T11:48:48.316471+0000","last_became_active":"2024-02-18T11:48:47.322708+0000","last_became_peered":"2024-02-18T11:48:47.322708+0000","last_unstale":"2024-02-18T11:48:48.316471+0000","last_undegraded":"2024-02-18T11:48:48.316471+0000","last_fullsized":"2024-02-18T11:48:48.316471+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:55:53.789541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315138+0000","last_change":"2024-02-18T11:48:47.314251+0000","last_active":"2024-02-18T11:48:48.315138+0000","last_peered":"2024-02-18T11:48:48.315138+0000","last_clean":"2024-02-18T11:48:48.315138+0000","last_became_active":"2024-02-18T11:48:47.313978+0000","last_became_peered":"2024-02-18T11:48:47.313978+0000","last_unstale":"2024-02-18T11:48:48.315138+0000","last_undegraded":"2024-02-18T11:48:48.315138+0000","last_fullsized":"2024-02-18T11:48:48.315138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:05.071355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316456+0000","last_change":"2024-02-18T11:48:47.315543+0000","last_active":"2024-02-18T11:48:48.316456+0000","last_peered":"2024-02-18T11:48:48.316456+0000","last_clean":"2024-02-18T11:48:48.316456+0000","last_became_active":"2024-02-18T11:48:47.315173+0000","last_became_peered":"2024-02-18T11:48:47.315173+0000","last_unstale":"2024-02-18T11:48:48.316456+0000","last_undegraded":"2024-02-18T11:48:48.316456+0000","last_fullsized":"2024-02-18T11:48:48.316456+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850281+0000","last_change":"2024-02-18T11:48:47.327854+0000","last_active":"2024-02-18T11:48:48.850281+0000","last_peered":"2024-02-18T11:48:48.850281+0000","last_clean":"2024-02-18T11:48:48.850281+0000","last_became_active":"2024-02-18T11:48:47.327598+0000","last_became_peered":"2024-02-18T11:48:47.327598+0000","last_unstale":"2024-02-18T11:48:48.850281+0000","last_undegraded":"2024-02-18T11:48:48.850281+0000","last_fullsized":"2024-02-18T11:48:48.850281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:55:35.888377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316406+0000","last_change":"2024-02-18T11:48:47.309300+0000","last_active":"2024-02-18T11:48:48.316406+0000","last_peered":"2024-02-18T11:48:48.316406+0000","last_clean":"2024-02-18T11:48:48.316406+0000","last_became_active":"2024-02-18T11:48:47.309025+0000","last_became_peered":"2024-02-18T11:48:47.309025+0000","last_unstale":"2024-02-18T11:48:48.316406+0000","last_undegraded":"2024-02-18T11:48:48.316406+0000","last_fullsized":"2024-02-18T11:48:48.316406+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:37.307682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850645+0000","last_change":"2024-02-18T11:48:47.329662+0000","last_active":"2024-02-18T11:48:48.850645+0000","last_peered":"2024-02-18T11:48:48.850645+0000","last_clean":"2024-02-18T11:48:48.850645+0000","last_became_active":"2024-02-18T11:48:47.329299+0000","last_became_peered":"2024-02-18T11:48:47.329299+0000","last_unstale":"2024-02-18T11:48:48.850645+0000","last_undegraded":"2024-02-18T11:48:48.850645+0000","last_fullsized":"2024-02-18T11:48:48.850645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:15:38.182081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316507+0000","last_change":"2024-02-18T11:48:47.317902+0000","last_active":"2024-02-18T11:48:48.316507+0000","last_peered":"2024-02-18T11:48:48.316507+0000","last_clean":"2024-02-18T11:48:48.316507+0000","last_became_active":"2024-02-18T11:48:47.317612+0000","last_became_peered":"2024-02-18T11:48:47.317612+0000","last_unstale":"2024-02-18T11:48:48.316507+0000","last_undegraded":"2024-02-18T11:48:48.316507+0000","last_fullsized":"2024-02-18T11:48:48.316507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316091+0000","last_change":"2024-02-18T11:48:47.311897+0000","last_active":"2024-02-18T11:48:48.316091+0000","last_peered":"2024-02-18T11:48:48.316091+0000","last_clean":"2024-02-18T11:48:48.316091+0000","last_became_active":"2024-02-18T11:48:47.311571+0000","last_became_peered":"2024-02-18T11:48:47.311571+0000","last_unstale":"2024-02-18T11:48:48.316091+0000","last_undegraded":"2024-02-18T11:48:48.316091+0000","last_fullsized":"2024-02-18T11:48:48.316091+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:45.431636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.317083+0000","last_change":"2024-02-18T11:48:47.323161+0000","last_active":"2024-02-18T11:48:48.317083+0000","last_peered":"2024-02-18T11:48:48.317083+0000","last_clean":"2024-02-18T11:48:48.317083+0000","last_became_active":"2024-02-18T11:48:47.322740+0000","last_became_peered":"2024-02-18T11:48:47.322740+0000","last_unstale":"2024-02-18T11:48:48.317083+0000","last_undegraded":"2024-02-18T11:48:48.317083+0000","last_fullsized":"2024-02-18T11:48:48.317083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:20:25.909108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849733+0000","last_change":"2024-02-18T11:48:47.315824+0000","last_active":"2024-02-18T11:48:48.849733+0000","last_peered":"2024-02-18T11:48:48.849733+0000","last_clean":"2024-02-18T11:48:48.849733+0000","last_became_active":"2024-02-18T11:48:47.315584+0000","last_became_peered":"2024-02-18T11:48:47.315584+0000","last_unstale":"2024-02-18T11:48:48.849733+0000","last_undegraded":"2024-02-18T11:48:48.849733+0000","last_fullsized":"2024-02-18T11:48:48.849733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316745+0000","last_change":"2024-02-18T11:48:47.321124+0000","last_active":"2024-02-18T11:48:48.316745+0000","last_peered":"2024-02-18T11:48:48.316745+0000","last_clean":"2024-02-18T11:48:48.316745+0000","last_became_active":"2024-02-18T11:48:47.320168+0000","last_became_peered":"2024-02-18T11:48:47.320168+0000","last_unstale":"2024-02-18T11:48:48.316745+0000","last_undegraded":"2024-02-18T11:48:48.316745+0000","last_fullsized":"2024-02-18T11:48:48.316745+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:51:54.332974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316439+0000","last_change":"2024-02-18T11:48:46.297461+0000","last_active":"2024-02-18T11:48:48.316439+0000","last_peered":"2024-02-18T11:48:48.316439+0000","last_clean":"2024-02-18T11:48:48.316439+0000","last_became_active":"2024-02-18T11:48:46.297166+0000","last_became_peered":"2024-02-18T11:48:46.297166+0000","last_unstale":"2024-02-18T11:48:48.316439+0000","last_undegraded":"2024-02-18T11:48:48.316439+0000","last_fullsized":"2024-02-18T11:48:48.316439+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850047+0000","last_change":"2024-02-18T11:48:47.326802+0000","last_active":"2024-02-18T11:48:48.850047+0000","last_peered":"2024-02-18T11:48:48.850047+0000","last_clean":"2024-02-18T11:48:48.850047+0000","last_became_active":"2024-02-18T11:48:47.326503+0000","last_became_peered":"2024-02-18T11:48:47.326503+0000","last_unstale":"2024-02-18T11:48:48.850047+0000","last_undegraded":"2024-02-18T11:48:48.850047+0000","last_fullsized":"2024-02-18T11:48:48.850047+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850010+0000","last_change":"2024-02-18T11:48:46.300302+0000","last_active":"2024-02-18T11:48:48.850010+0000","last_peered":"2024-02-18T11:48:48.850010+0000","last_clean":"2024-02-18T11:48:48.850010+0000","last_became_active":"2024-02-18T11:48:46.299909+0000","last_became_peered":"2024-02-18T11:48:46.299909+0000","last_unstale":"2024-02-18T11:48:48.850010+0000","last_undegraded":"2024-02-18T11:48:48.850010+0000","last_fullsized":"2024-02-18T11:48:48.850010+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850675+0000","last_change":"2024-02-18T11:48:47.316579+0000","last_active":"2024-02-18T11:48:48.850675+0000","last_peered":"2024-02-18T11:48:48.850675+0000","last_clean":"2024-02-18T11:48:48.850675+0000","last_became_active":"2024-02-18T11:48:47.316195+0000","last_became_peered":"2024-02-18T11:48:47.316195+0000","last_unstale":"2024-02-18T11:48:48.850675+0000","last_undegraded":"2024-02-18T11:48:48.850675+0000","last_fullsized":"2024-02-18T11:48:48.850675+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850098+0000","last_change":"2024-02-18T11:48:46.297756+0000","last_active":"2024-02-18T11:48:48.850098+0000","last_peered":"2024-02-18T11:48:48.850098+0000","last_clean":"2024-02-18T11:48:48.850098+0000","last_became_active":"2024-02-18T11:48:46.297485+0000","last_became_peered":"2024-02-18T11:48:46.297485+0000","last_unstale":"2024-02-18T11:48:48.850098+0000","last_undegraded":"2024-02-18T11:48:48.850098+0000","last_fullsized":"2024-02-18T11:48:48.850098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T12:57:17.857429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850781+0000","last_change":"2024-02-18T11:48:47.317072+0000","last_active":"2024-02-18T11:48:48.850781+0000","last_peered":"2024-02-18T11:48:48.850781+0000","last_clean":"2024-02-18T11:48:48.850781+0000","last_became_active":"2024-02-18T11:48:47.316360+0000","last_became_peered":"2024-02-18T11:48:47.316360+0000","last_unstale":"2024-02-18T11:48:48.850781+0000","last_undegraded":"2024-02-18T11:48:48.850781+0000","last_fullsized":"2024-02-18T11:48:48.850781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:13:45.002912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850646+0000","last_change":"2024-02-18T11:48:46.305539+0000","last_active":"2024-02-18T11:48:48.850646+0000","last_peered":"2024-02-18T11:48:48.850646+0000","last_clean":"2024-02-18T11:48:48.850646+0000","last_became_active":"2024-02-18T11:48:46.305252+0000","last_became_peered":"2024-02-18T11:48:46.305252+0000","last_unstale":"2024-02-18T11:48:48.850646+0000","last_undegraded":"2024-02-18T11:48:48.850646+0000","last_fullsized":"2024-02-18T11:48:48.850646+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849547+0000","last_change":"2024-02-18T11:48:47.321546+0000","last_active":"2024-02-18T11:48:48.849547+0000","last_peered":"2024-02-18T11:48:48.849547+0000","last_clean":"2024-02-18T11:48:48.849547+0000","last_became_active":"2024-02-18T11:48:47.321288+0000","last_became_peered":"2024-02-18T11:48:47.321288+0000","last_unstale":"2024-02-18T11:48:48.849547+0000","last_undegraded":"2024-02-18T11:48:48.849547+0000","last_fullsized":"2024-02-18T11:48:48.849547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:57:10.461952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850418+0000","last_change":"2024-02-18T11:48:46.308595+0000","last_active":"2024-02-18T11:48:48.850418+0000","last_peered":"2024-02-18T11:48:48.850418+0000","last_clean":"2024-02-18T11:48:48.850418+0000","last_became_active":"2024-02-18T11:48:46.308361+0000","last_became_peered":"2024-02-18T11:48:46.308361+0000","last_unstale":"2024-02-18T11:48:48.850418+0000","last_undegraded":"2024-02-18T11:48:48.850418+0000","last_fullsized":"2024-02-18T11:48:48.850418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850792+0000","last_change":"2024-02-18T11:48:47.327262+0000","last_active":"2024-02-18T11:48:48.850792+0000","last_peered":"2024-02-18T11:48:48.850792+0000","last_clean":"2024-02-18T11:48:48.850792+0000","last_became_active":"2024-02-18T11:48:47.326914+0000","last_became_peered":"2024-02-18T11:48:47.326914+0000","last_unstale":"2024-02-18T11:48:48.850792+0000","last_undegraded":"2024-02-18T11:48:48.850792+0000","last_fullsized":"2024-02-18T11:48:48.850792+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:11:58.023116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.317159+0000","last_change":"2024-02-18T11:48:46.300103+0000","last_active":"2024-02-18T11:48:48.317159+0000","last_peered":"2024-02-18T11:48:48.317159+0000","last_clean":"2024-02-18T11:48:48.317159+0000","last_became_active":"2024-02-18T11:48:46.299808+0000","last_became_peered":"2024-02-18T11:48:46.299808+0000","last_unstale":"2024-02-18T11:48:48.317159+0000","last_undegraded":"2024-02-18T11:48:48.317159+0000","last_fullsized":"2024-02-18T11:48:48.317159+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850855+0000","last_change":"2024-02-18T11:48:47.320639+0000","last_active":"2024-02-18T11:48:48.850855+0000","last_peered":"2024-02-18T11:48:48.850855+0000","last_clean":"2024-02-18T11:48:48.850855+0000","last_became_active":"2024-02-18T11:48:47.320385+0000","last_became_peered":"2024-02-18T11:48:47.320385+0000","last_unstale":"2024-02-18T11:48:48.850855+0000","last_undegraded":"2024-02-18T11:48:48.850855+0000","last_fullsized":"2024-02-18T11:48:48.850855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:51.558721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850196+0000","last_change":"2024-02-18T11:48:46.304701+0000","last_active":"2024-02-18T11:48:48.850196+0000","last_peered":"2024-02-18T11:48:48.850196+0000","last_clean":"2024-02-18T11:48:48.850196+0000","last_became_active":"2024-02-18T11:48:46.304450+0000","last_became_peered":"2024-02-18T11:48:46.304450+0000","last_unstale":"2024-02-18T11:48:48.850196+0000","last_undegraded":"2024-02-18T11:48:48.850196+0000","last_fullsized":"2024-02-18T11:48:48.850196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850534+0000","last_change":"2024-02-18T11:48:47.317646+0000","last_active":"2024-02-18T11:48:48.850534+0000","last_peered":"2024-02-18T11:48:48.850534+0000","last_clean":"2024-02-18T11:48:48.850534+0000","last_became_active":"2024-02-18T11:48:47.317171+0000","last_became_peered":"2024-02-18T11:48:47.317171+0000","last_unstale":"2024-02-18T11:48:48.850534+0000","last_undegraded":"2024-02-18T11:48:48.850534+0000","last_fullsized":"2024-02-18T11:48:48.850534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:39:17.095580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850019+0000","last_change":"2024-02-18T11:48:46.296450+0000","last_active":"2024-02-18T11:48:48.850019+0000","last_peered":"2024-02-18T11:48:48.850019+0000","last_clean":"2024-02-18T11:48:48.850019+0000","last_became_active":"2024-02-18T11:48:46.296159+0000","last_became_peered":"2024-02-18T11:48:46.296159+0000","last_unstale":"2024-02-18T11:48:48.850019+0000","last_undegraded":"2024-02-18T11:48:48.850019+0000","last_fullsized":"2024-02-18T11:48:48.850019+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315534+0000","last_change":"2024-02-18T11:48:47.325933+0000","last_active":"2024-02-18T11:48:48.315534+0000","last_peered":"2024-02-18T11:48:48.315534+0000","last_clean":"2024-02-18T11:48:48.315534+0000","last_became_active":"2024-02-18T11:48:47.325686+0000","last_became_peered":"2024-02-18T11:48:47.325686+0000","last_unstale":"2024-02-18T11:48:48.315534+0000","last_undegraded":"2024-02-18T11:48:48.315534+0000","last_fullsized":"2024-02-18T11:48:48.315534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:22.140759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850835+0000","last_change":"2024-02-18T11:48:46.306411+0000","last_active":"2024-02-18T11:48:48.850835+0000","last_peered":"2024-02-18T11:48:48.850835+0000","last_clean":"2024-02-18T11:48:48.850835+0000","last_became_active":"2024-02-18T11:48:46.304718+0000","last_became_peered":"2024-02-18T11:48:46.304718+0000","last_unstale":"2024-02-18T11:48:48.850835+0000","last_undegraded":"2024-02-18T11:48:48.850835+0000","last_fullsized":"2024-02-18T11:48:48.850835+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:57:22.163523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.317071+0000","last_change":"2024-02-18T11:48:47.324227+0000","last_active":"2024-02-18T11:48:48.317071+0000","last_peered":"2024-02-18T11:48:48.317071+0000","last_clean":"2024-02-18T11:48:48.317071+0000","last_became_active":"2024-02-18T11:48:47.323968+0000","last_became_peered":"2024-02-18T11:48:47.323968+0000","last_unstale":"2024-02-18T11:48:48.317071+0000","last_undegraded":"2024-02-18T11:48:48.317071+0000","last_fullsized":"2024-02-18T11:48:48.317071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:52:59.755152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851052+0000","last_change":"2024-02-18T11:48:46.307561+0000","last_active":"2024-02-18T11:48:48.851052+0000","last_peered":"2024-02-18T11:48:48.851052+0000","last_clean":"2024-02-18T11:48:48.851052+0000","last_became_active":"2024-02-18T11:48:46.307253+0000","last_became_peered":"2024-02-18T11:48:46.307253+0000","last_unstale":"2024-02-18T11:48:48.851052+0000","last_undegraded":"2024-02-18T11:48:48.851052+0000","last_fullsized":"2024-02-18T11:48:48.851052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849994+0000","last_change":"2024-02-18T11:48:47.316738+0000","last_active":"2024-02-18T11:48:48.849994+0000","last_peered":"2024-02-18T11:48:48.849994+0000","last_clean":"2024-02-18T11:48:48.849994+0000","last_became_active":"2024-02-18T11:48:47.316488+0000","last_became_peered":"2024-02-18T11:48:47.316488+0000","last_unstale":"2024-02-18T11:48:48.849994+0000","last_undegraded":"2024-02-18T11:48:48.849994+0000","last_fullsized":"2024-02-18T11:48:48.849994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:29:29.908143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851193+0000","last_change":"2024-02-18T11:48:46.304153+0000","last_active":"2024-02-18T11:48:48.851193+0000","last_peered":"2024-02-18T11:48:48.851193+0000","last_clean":"2024-02-18T11:48:48.851193+0000","last_became_active":"2024-02-18T11:48:46.303687+0000","last_became_peered":"2024-02-18T11:48:46.303687+0000","last_unstale":"2024-02-18T11:48:48.851193+0000","last_undegraded":"2024-02-18T11:48:48.851193+0000","last_fullsized":"2024-02-18T11:48:48.851193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850604+0000","last_change":"2024-02-18T11:48:47.324065+0000","last_active":"2024-02-18T11:48:48.850604+0000","last_peered":"2024-02-18T11:48:48.850604+0000","last_clean":"2024-02-18T11:48:48.850604+0000","last_became_active":"2024-02-18T11:48:47.323797+0000","last_became_peered":"2024-02-18T11:48:47.323797+0000","last_unstale":"2024-02-18T11:48:48.850604+0000","last_undegraded":"2024-02-18T11:48:48.850604+0000","last_fullsized":"2024-02-18T11:48:48.850604+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:33.994524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850771+0000","last_change":"2024-02-18T11:48:46.301330+0000","last_active":"2024-02-18T11:48:48.850771+0000","last_peered":"2024-02-18T11:48:48.850771+0000","last_clean":"2024-02-18T11:48:48.850771+0000","last_became_active":"2024-02-18T11:48:46.301062+0000","last_became_peered":"2024-02-18T11:48:46.301062+0000","last_unstale":"2024-02-18T11:48:48.850771+0000","last_undegraded":"2024-02-18T11:48:48.850771+0000","last_fullsized":"2024-02-18T11:48:48.850771+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:36.303932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849568+0000","last_change":"2024-02-18T11:48:47.317238+0000","last_active":"2024-02-18T11:48:48.849568+0000","last_peered":"2024-02-18T11:48:48.849568+0000","last_clean":"2024-02-18T11:48:48.849568+0000","last_became_active":"2024-02-18T11:48:47.316957+0000","last_became_peered":"2024-02-18T11:48:47.316957+0000","last_unstale":"2024-02-18T11:48:48.849568+0000","last_undegraded":"2024-02-18T11:48:48.849568+0000","last_fullsized":"2024-02-18T11:48:48.849568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:15:45.332480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850707+0000","last_change":"2024-02-18T11:48:46.307286+0000","last_active":"2024-02-18T11:48:48.850707+0000","last_peered":"2024-02-18T11:48:48.850707+0000","last_clean":"2024-02-18T11:48:48.850707+0000","last_became_active":"2024-02-18T11:48:46.307002+0000","last_became_peered":"2024-02-18T11:48:46.307002+0000","last_unstale":"2024-02-18T11:48:48.850707+0000","last_undegraded":"2024-02-18T11:48:48.850707+0000","last_fullsized":"2024-02-18T11:48:48.850707+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:21:21.137008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850462+0000","last_change":"2024-02-18T11:48:47.323364+0000","last_active":"2024-02-18T11:48:48.850462+0000","last_peered":"2024-02-18T11:48:48.850462+0000","last_clean":"2024-02-18T11:48:48.850462+0000","last_became_active":"2024-02-18T11:48:47.318734+0000","last_became_peered":"2024-02-18T11:48:47.318734+0000","last_unstale":"2024-02-18T11:48:48.850462+0000","last_undegraded":"2024-02-18T11:48:48.850462+0000","last_fullsized":"2024-02-18T11:48:48.850462+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:08:35.000436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849878+0000","last_change":"2024-02-18T11:48:46.305434+0000","last_active":"2024-02-18T11:48:48.849878+0000","last_peered":"2024-02-18T11:48:48.849878+0000","last_clean":"2024-02-18T11:48:48.849878+0000","last_became_active":"2024-02-18T11:48:46.305145+0000","last_became_peered":"2024-02-18T11:48:46.305145+0000","last_unstale":"2024-02-18T11:48:48.849878+0000","last_undegraded":"2024-02-18T11:48:48.849878+0000","last_fullsized":"2024-02-18T11:48:48.849878+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:47:07.775308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316316+0000","last_change":"2024-02-18T11:48:47.320430+0000","last_active":"2024-02-18T11:48:48.316316+0000","last_peered":"2024-02-18T11:48:48.316316+0000","last_clean":"2024-02-18T11:48:48.316316+0000","last_became_active":"2024-02-18T11:48:47.320153+0000","last_became_peered":"2024-02-18T11:48:47.320153+0000","last_unstale":"2024-02-18T11:48:48.316316+0000","last_undegraded":"2024-02-18T11:48:48.316316+0000","last_fullsized":"2024-02-18T11:48:48.316316+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:29:29.908143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315838+0000","last_change":"2024-02-18T11:48:46.297664+0000","last_active":"2024-02-18T11:48:48.315838+0000","last_peered":"2024-02-18T11:48:48.315838+0000","last_clean":"2024-02-18T11:48:48.315838+0000","last_became_active":"2024-02-18T11:48:46.297202+0000","last_became_peered":"2024-02-18T11:48:46.297202+0000","last_unstale":"2024-02-18T11:48:48.315838+0000","last_undegraded":"2024-02-18T11:48:48.315838+0000","last_fullsized":"2024-02-18T11:48:48.315838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850622+0000","last_change":"2024-02-18T11:48:47.318448+0000","last_active":"2024-02-18T11:48:48.850622+0000","last_peered":"2024-02-18T11:48:48.850622+0000","last_clean":"2024-02-18T11:48:48.850622+0000","last_became_active":"2024-02-18T11:48:47.318190+0000","last_became_peered":"2024-02-18T11:48:47.318190+0000","last_unstale":"2024-02-18T11:48:48.850622+0000","last_undegraded":"2024-02-18T11:48:48.850622+0000","last_fullsized":"2024-02-18T11:48:48.850622+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:54.435871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316594+0000","last_change":"2024-02-18T11:48:46.300667+0000","last_active":"2024-02-18T11:48:48.316594+0000","last_peered":"2024-02-18T11:48:48.316594+0000","last_clean":"2024-02-18T11:48:48.316594+0000","last_became_active":"2024-02-18T11:48:46.300407+0000","last_became_peered":"2024-02-18T11:48:46.300407+0000","last_unstale":"2024-02-18T11:48:48.316594+0000","last_undegraded":"2024-02-18T11:48:48.316594+0000","last_fullsized":"2024-02-18T11:48:48.316594+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850687+0000","last_change":"2024-02-18T11:48:47.309612+0000","last_active":"2024-02-18T11:48:48.850687+0000","last_peered":"2024-02-18T11:48:48.850687+0000","last_clean":"2024-02-18T11:48:48.850687+0000","last_became_active":"2024-02-18T11:48:47.309382+0000","last_became_peered":"2024-02-18T11:48:47.309382+0000","last_unstale":"2024-02-18T11:48:48.850687+0000","last_undegraded":"2024-02-18T11:48:48.850687+0000","last_fullsized":"2024-02-18T11:48:48.850687+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:51:54.332974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851037+0000","last_change":"2024-02-18T11:48:46.307646+0000","last_active":"2024-02-18T11:48:48.851037+0000","last_peered":"2024-02-18T11:48:48.851037+0000","last_clean":"2024-02-18T11:48:48.851037+0000","last_became_active":"2024-02-18T11:48:46.307345+0000","last_became_peered":"2024-02-18T11:48:46.307345+0000","last_unstale":"2024-02-18T11:48:48.851037+0000","last_undegraded":"2024-02-18T11:48:48.851037+0000","last_fullsized":"2024-02-18T11:48:48.851037+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:53:50.554971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851220+0000","last_change":"2024-02-18T11:48:47.327157+0000","last_active":"2024-02-18T11:48:48.851220+0000","last_peered":"2024-02-18T11:48:48.851220+0000","last_clean":"2024-02-18T11:48:48.851220+0000","last_became_active":"2024-02-18T11:48:47.326879+0000","last_became_peered":"2024-02-18T11:48:47.326879+0000","last_unstale":"2024-02-18T11:48:48.851220+0000","last_undegraded":"2024-02-18T11:48:48.851220+0000","last_fullsized":"2024-02-18T11:48:48.851220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:09:34.514561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850596+0000","last_change":"2024-02-18T11:48:46.306743+0000","last_active":"2024-02-18T11:48:48.850596+0000","last_peered":"2024-02-18T11:48:48.850596+0000","last_clean":"2024-02-18T11:48:48.850596+0000","last_became_active":"2024-02-18T11:48:46.306447+0000","last_became_peered":"2024-02-18T11:48:46.306447+0000","last_unstale":"2024-02-18T11:48:48.850596+0000","last_undegraded":"2024-02-18T11:48:48.850596+0000","last_fullsized":"2024-02-18T11:48:48.850596+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850365+0000","last_change":"2024-02-18T11:48:47.321271+0000","last_active":"2024-02-18T11:48:48.850365+0000","last_peered":"2024-02-18T11:48:48.850365+0000","last_clean":"2024-02-18T11:48:48.850365+0000","last_became_active":"2024-02-18T11:48:47.321021+0000","last_became_peered":"2024-02-18T11:48:47.321021+0000","last_unstale":"2024-02-18T11:48:48.850365+0000","last_undegraded":"2024-02-18T11:48:48.850365+0000","last_fullsized":"2024-02-18T11:48:48.850365+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:11:37.307682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315372+0000","last_change":"2024-02-18T11:48:46.296023+0000","last_active":"2024-02-18T11:48:48.315372+0000","last_peered":"2024-02-18T11:48:48.315372+0000","last_clean":"2024-02-18T11:48:48.315372+0000","last_became_active":"2024-02-18T11:48:46.295715+0000","last_became_peered":"2024-02-18T11:48:46.295715+0000","last_unstale":"2024-02-18T11:48:48.315372+0000","last_undegraded":"2024-02-18T11:48:48.315372+0000","last_fullsized":"2024-02-18T11:48:48.315372+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315418+0000","last_change":"2024-02-18T11:48:47.310131+0000","last_active":"2024-02-18T11:48:48.315418+0000","last_peered":"2024-02-18T11:48:48.315418+0000","last_clean":"2024-02-18T11:48:48.315418+0000","last_became_active":"2024-02-18T11:48:47.309877+0000","last_became_peered":"2024-02-18T11:48:47.309877+0000","last_unstale":"2024-02-18T11:48:48.315418+0000","last_undegraded":"2024-02-18T11:48:48.315418+0000","last_fullsized":"2024-02-18T11:48:48.315418+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:34.883282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849732+0000","last_change":"2024-02-18T11:48:46.297720+0000","last_active":"2024-02-18T11:48:48.849732+0000","last_peered":"2024-02-18T11:48:48.849732+0000","last_clean":"2024-02-18T11:48:48.849732+0000","last_became_active":"2024-02-18T11:48:46.297462+0000","last_became_peered":"2024-02-18T11:48:46.297462+0000","last_unstale":"2024-02-18T11:48:48.849732+0000","last_undegraded":"2024-02-18T11:48:48.849732+0000","last_fullsized":"2024-02-18T11:48:48.849732+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:21:45.610109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316830+0000","last_change":"2024-02-18T11:48:47.321057+0000","last_active":"2024-02-18T11:48:48.316830+0000","last_peered":"2024-02-18T11:48:48.316830+0000","last_clean":"2024-02-18T11:48:48.316830+0000","last_became_active":"2024-02-18T11:48:47.320573+0000","last_became_peered":"2024-02-18T11:48:47.320573+0000","last_unstale":"2024-02-18T11:48:48.316830+0000","last_undegraded":"2024-02-18T11:48:48.316830+0000","last_fullsized":"2024-02-18T11:48:48.316830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T22:08:35.000436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315297+0000","last_change":"2024-02-18T11:48:46.303136+0000","last_active":"2024-02-18T11:48:48.315297+0000","last_peered":"2024-02-18T11:48:48.315297+0000","last_clean":"2024-02-18T11:48:48.315297+0000","last_became_active":"2024-02-18T11:48:46.302861+0000","last_became_peered":"2024-02-18T11:48:46.302861+0000","last_unstale":"2024-02-18T11:48:48.315297+0000","last_undegraded":"2024-02-18T11:48:48.315297+0000","last_fullsized":"2024-02-18T11:48:48.315297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:11:23.704773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849684+0000","last_change":"2024-02-18T11:48:47.319702+0000","last_active":"2024-02-18T11:48:48.849684+0000","last_peered":"2024-02-18T11:48:48.849684+0000","last_clean":"2024-02-18T11:48:48.849684+0000","last_became_active":"2024-02-18T11:48:47.319464+0000","last_became_peered":"2024-02-18T11:48:47.319464+0000","last_unstale":"2024-02-18T11:48:48.849684+0000","last_undegraded":"2024-02-18T11:48:48.849684+0000","last_fullsized":"2024-02-18T11:48:48.849684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T15:55:35.888377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315380+0000","last_change":"2024-02-18T11:48:46.297761+0000","last_active":"2024-02-18T11:48:48.315380+0000","last_peered":"2024-02-18T11:48:48.315380+0000","last_clean":"2024-02-18T11:48:48.315380+0000","last_became_active":"2024-02-18T11:48:46.297500+0000","last_became_peered":"2024-02-18T11:48:46.297500+0000","last_unstale":"2024-02-18T11:48:48.315380+0000","last_undegraded":"2024-02-18T11:48:48.315380+0000","last_fullsized":"2024-02-18T11:48:48.315380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:33:10.032538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850431+0000","last_change":"2024-02-18T11:48:47.310081+0000","last_active":"2024-02-18T11:48:48.850431+0000","last_peered":"2024-02-18T11:48:48.850431+0000","last_clean":"2024-02-18T11:48:48.850431+0000","last_became_active":"2024-02-18T11:48:47.309813+0000","last_became_peered":"2024-02-18T11:48:47.309813+0000","last_unstale":"2024-02-18T11:48:48.850431+0000","last_undegraded":"2024-02-18T11:48:48.850431+0000","last_fullsized":"2024-02-18T11:48:48.850431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:20:25.909108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316601+0000","last_change":"2024-02-18T11:48:46.298569+0000","last_active":"2024-02-18T11:48:48.316601+0000","last_peered":"2024-02-18T11:48:48.316601+0000","last_clean":"2024-02-18T11:48:48.316601+0000","last_became_active":"2024-02-18T11:48:46.297865+0000","last_became_peered":"2024-02-18T11:48:46.297865+0000","last_unstale":"2024-02-18T11:48:48.316601+0000","last_undegraded":"2024-02-18T11:48:48.316601+0000","last_fullsized":"2024-02-18T11:48:48.316601+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:37:04.067605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850672+0000","last_change":"2024-02-18T11:48:47.312852+0000","last_active":"2024-02-18T11:48:48.850672+0000","last_peered":"2024-02-18T11:48:48.850672+0000","last_clean":"2024-02-18T11:48:48.850672+0000","last_became_active":"2024-02-18T11:48:47.312584+0000","last_became_peered":"2024-02-18T11:48:47.312584+0000","last_unstale":"2024-02-18T11:48:48.850672+0000","last_undegraded":"2024-02-18T11:48:48.850672+0000","last_fullsized":"2024-02-18T11:48:48.850672+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:52:59.755152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315078+0000","last_change":"2024-02-18T11:48:46.302429+0000","last_active":"2024-02-18T11:48:48.315078+0000","last_peered":"2024-02-18T11:48:48.315078+0000","last_clean":"2024-02-18T11:48:48.315078+0000","last_became_active":"2024-02-18T11:48:46.302138+0000","last_became_peered":"2024-02-18T11:48:46.302138+0000","last_unstale":"2024-02-18T11:48:48.315078+0000","last_undegraded":"2024-02-18T11:48:48.315078+0000","last_fullsized":"2024-02-18T11:48:48.315078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:29:54.091351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849941+0000","last_change":"2024-02-18T11:48:47.320342+0000","last_active":"2024-02-18T11:48:48.849941+0000","last_peered":"2024-02-18T11:48:48.849941+0000","last_clean":"2024-02-18T11:48:48.849941+0000","last_became_active":"2024-02-18T11:48:47.319996+0000","last_became_peered":"2024-02-18T11:48:47.319996+0000","last_unstale":"2024-02-18T11:48:48.849941+0000","last_undegraded":"2024-02-18T11:48:48.849941+0000","last_fullsized":"2024-02-18T11:48:48.849941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:14.550912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851193+0000","last_change":"2024-02-18T11:48:46.308271+0000","last_active":"2024-02-18T11:48:48.851193+0000","last_peered":"2024-02-18T11:48:48.851193+0000","last_clean":"2024-02-18T11:48:48.851193+0000","last_became_active":"2024-02-18T11:48:46.307981+0000","last_became_peered":"2024-02-18T11:48:46.307981+0000","last_unstale":"2024-02-18T11:48:48.851193+0000","last_undegraded":"2024-02-18T11:48:48.851193+0000","last_fullsized":"2024-02-18T11:48:48.851193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:23:32.990774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.317041+0000","last_change":"2024-02-18T11:48:47.324838+0000","last_active":"2024-02-18T11:48:48.317041+0000","last_peered":"2024-02-18T11:48:48.317041+0000","last_clean":"2024-02-18T11:48:48.317041+0000","last_became_active":"2024-02-18T11:48:47.324576+0000","last_became_peered":"2024-02-18T11:48:47.324576+0000","last_unstale":"2024-02-18T11:48:48.317041+0000","last_undegraded":"2024-02-18T11:48:48.317041+0000","last_fullsized":"2024-02-18T11:48:48.317041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T21:13:54.435871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.315310+0000","last_change":"2024-02-18T11:48:46.305551+0000","last_active":"2024-02-18T11:48:48.315310+0000","last_peered":"2024-02-18T11:48:48.315310+0000","last_clean":"2024-02-18T11:48:48.315310+0000","last_became_active":"2024-02-18T11:48:46.305246+0000","last_became_peered":"2024-02-18T11:48:46.305246+0000","last_unstale":"2024-02-18T11:48:48.315310+0000","last_undegraded":"2024-02-18T11:48:48.315310+0000","last_fullsized":"2024-02-18T11:48:48.315310+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:26:45.174946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850748+0000","last_change":"2024-02-18T11:48:47.319083+0000","last_active":"2024-02-18T11:48:48.850748+0000","last_peered":"2024-02-18T11:48:48.850748+0000","last_clean":"2024-02-18T11:48:48.850748+0000","last_became_active":"2024-02-18T11:48:47.318762+0000","last_became_peered":"2024-02-18T11:48:47.318762+0000","last_unstale":"2024-02-18T11:48:48.850748+0000","last_undegraded":"2024-02-18T11:48:48.850748+0000","last_fullsized":"2024-02-18T11:48:48.850748+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T14:15:38.182081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849925+0000","last_change":"2024-02-18T11:48:46.305494+0000","last_active":"2024-02-18T11:48:48.849925+0000","last_peered":"2024-02-18T11:48:48.849925+0000","last_clean":"2024-02-18T11:48:48.849925+0000","last_became_active":"2024-02-18T11:48:46.305222+0000","last_became_peered":"2024-02-18T11:48:46.305222+0000","last_unstale":"2024-02-18T11:48:48.849925+0000","last_undegraded":"2024-02-18T11:48:48.849925+0000","last_fullsized":"2024-02-18T11:48:48.849925+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T23:46:21.769287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.849698+0000","last_change":"2024-02-18T11:48:47.324927+0000","last_active":"2024-02-18T11:48:48.849698+0000","last_peered":"2024-02-18T11:48:48.849698+0000","last_clean":"2024-02-18T11:48:48.849698+0000","last_became_active":"2024-02-18T11:48:47.324596+0000","last_became_peered":"2024-02-18T11:48:47.324596+0000","last_unstale":"2024-02-18T11:48:48.849698+0000","last_undegraded":"2024-02-18T11:48:48.849698+0000","last_fullsized":"2024-02-18T11:48:48.849698+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:03:08.526820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850154+0000","last_change":"2024-02-18T11:48:46.303722+0000","last_active":"2024-02-18T11:48:48.850154+0000","last_peered":"2024-02-18T11:48:48.850154+0000","last_clean":"2024-02-18T11:48:48.850154+0000","last_became_active":"2024-02-18T11:48:46.303460+0000","last_became_peered":"2024-02-18T11:48:46.303460+0000","last_unstale":"2024-02-18T11:48:48.850154+0000","last_undegraded":"2024-02-18T11:48:48.850154+0000","last_fullsized":"2024-02-18T11:48:48.850154+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:32:56.140704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850121+0000","last_change":"2024-02-18T11:48:47.320481+0000","last_active":"2024-02-18T11:48:48.850121+0000","last_peered":"2024-02-18T11:48:48.850121+0000","last_clean":"2024-02-18T11:48:48.850121+0000","last_became_active":"2024-02-18T11:48:47.320158+0000","last_became_peered":"2024-02-18T11:48:47.320158+0000","last_unstale":"2024-02-18T11:48:48.850121+0000","last_undegraded":"2024-02-18T11:48:48.850121+0000","last_fullsized":"2024-02-18T11:48:48.850121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:15:45.332480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316055+0000","last_change":"2024-02-18T11:48:46.299505+0000","last_active":"2024-02-18T11:48:48.316055+0000","last_peered":"2024-02-18T11:48:48.316055+0000","last_clean":"2024-02-18T11:48:48.316055+0000","last_became_active":"2024-02-18T11:48:46.299230+0000","last_became_peered":"2024-02-18T11:48:46.299230+0000","last_unstale":"2024-02-18T11:48:48.316055+0000","last_undegraded":"2024-02-18T11:48:48.316055+0000","last_fullsized":"2024-02-18T11:48:48.316055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:42:04.371627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.851070+0000","last_change":"2024-02-18T11:48:47.315114+0000","last_active":"2024-02-18T11:48:48.851070+0000","last_peered":"2024-02-18T11:48:48.851070+0000","last_clean":"2024-02-18T11:48:48.851070+0000","last_became_active":"2024-02-18T11:48:47.313064+0000","last_became_peered":"2024-02-18T11:48:47.313064+0000","last_unstale":"2024-02-18T11:48:48.851070+0000","last_undegraded":"2024-02-18T11:48:48.851070+0000","last_fullsized":"2024-02-18T11:48:48.851070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:12:14.550912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850075+0000","last_change":"2024-02-18T11:48:46.306424+0000","last_active":"2024-02-18T11:48:48.850075+0000","last_peered":"2024-02-18T11:48:48.850075+0000","last_clean":"2024-02-18T11:48:48.850075+0000","last_became_active":"2024-02-18T11:48:46.305938+0000","last_became_peered":"2024-02-18T11:48:46.305938+0000","last_unstale":"2024-02-18T11:48:48.850075+0000","last_undegraded":"2024-02-18T11:48:48.850075+0000","last_fullsized":"2024-02-18T11:48:48.850075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T18:08:48.967965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316960+0000","last_change":"2024-02-18T11:48:47.314001+0000","last_active":"2024-02-18T11:48:48.316960+0000","last_peered":"2024-02-18T11:48:48.316960+0000","last_clean":"2024-02-18T11:48:48.316960+0000","last_became_active":"2024-02-18T11:48:47.313680+0000","last_became_peered":"2024-02-18T11:48:47.313680+0000","last_unstale":"2024-02-18T11:48:48.316960+0000","last_undegraded":"2024-02-18T11:48:48.316960+0000","last_fullsized":"2024-02-18T11:48:48.316960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T13:39:17.095580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850288+0000","last_change":"2024-02-18T11:48:46.306107+0000","last_active":"2024-02-18T11:48:48.850288+0000","last_peered":"2024-02-18T11:48:48.850288+0000","last_clean":"2024-02-18T11:48:48.850288+0000","last_became_active":"2024-02-18T11:48:46.305749+0000","last_became_peered":"2024-02-18T11:48:46.305749+0000","last_unstale":"2024-02-18T11:48:48.850288+0000","last_undegraded":"2024-02-18T11:48:48.850288+0000","last_fullsized":"2024-02-18T11:48:48.850288+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T17:46:33.879531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.850913+0000","last_change":"2024-02-18T11:48:47.329459+0000","last_active":"2024-02-18T11:48:48.850913+0000","last_peered":"2024-02-18T11:48:48.850913+0000","last_clean":"2024-02-18T11:48:48.850913+0000","last_became_active":"2024-02-18T11:48:47.329195+0000","last_became_peered":"2024-02-18T11:48:47.329195+0000","last_unstale":"2024-02-18T11:48:48.850913+0000","last_undegraded":"2024-02-18T11:48:48.850913+0000","last_fullsized":"2024-02-18T11:48:48.850913+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:46.281573+0000","last_clean_scrub_stamp":"2024-02-18T11:48:46.281573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T16:48:38.514367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316813+0000","last_change":"2024-02-18T11:48:46.303447+0000","last_active":"2024-02-18T11:48:48.316813+0000","last_peered":"2024-02-18T11:48:48.316813+0000","last_clean":"2024-02-18T11:48:48.316813+0000","last_became_active":"2024-02-18T11:48:46.303126+0000","last_became_peered":"2024-02-18T11:48:46.303126+0000","last_unstale":"2024-02-18T11:48:48.316813+0000","last_undegraded":"2024-02-18T11:48:48.316813+0000","last_fullsized":"2024-02-18T11:48:48.316813+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:48:45.277823+0000","last_clean_scrub_stamp":"2024-02-18T11:48:45.277823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T19:25:44.427886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-02-18T11:48:48.316749+0000","last_change":"2024-02-18T11:47:23.428327+0000","last_active":"2024-02-18T11:48:48.316749+0000","last_peered":"2024-02-18T11:48:48.316749+0000","last_clean":"2024-02-18T11:48:48.316749+0000","last_became_active":"2024-02-18T11:47:23.427367+0000","last_became_peered":"2024-02-18T11:47:23.427367+0000","last_unstale":"2024-02-18T11:48:48.316749+0000","last_undegraded":"2024-02-18T11:48:48.316749+0000","last_fullsized":"2024-02-18T11:48:48.316749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T11:47:22.413113+0000","last_clean_scrub_stamp":"2024-02-18T11:47:22.413113+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-19T20:19:30.646215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":782240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":190302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":190794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":191836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":1036288,"data_stored":780592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T11:48:54.957 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-18T11:48:54.957 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-02-18T11:48:55.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.106+0000 7f26a218c640 1 Processor -- start 2024-02-18T11:48:55.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f26a218c640 1 -- start start 2024-02-18T11:48:55.107 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f26a218c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f269c0711c0 0x7f269c071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f26a218c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f269c15da30 0x7f269c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f26a218c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c15c960 con 0x7f269c145230 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f26a218c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c15d050 con 0x7f269c0711c0 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f26a218c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c145610 con 0x7f269c15da30 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f269affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f269c0711c0 0x7f269c071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f269affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f269c0711c0 0x7f269c071650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42030/0 (socket says 172.21.15.116:42030) 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.107+0000 7f269affd640 1 -- 172.21.15.116:0/2348858613 learned_addr learned my addr 172.21.15.116:0/2348858613 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.108+0000 7f269bfff640 1 --2- 172.21.15.116:0/2348858613 >> v2:172.21.15.143:3300/0 conn(0x7f269c15da30 0x7f269c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.108+0000 7f269affd640 1 -- 172.21.15.116:0/2348858613 >> v2:172.21.15.143:3300/0 conn(0x7f269c15da30 msgr2=0x7f269c15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:55.108 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.108+0000 7f269affd640 1 --2- 172.21.15.116:0/2348858613 >> v2:172.21.15.143:3300/0 conn(0x7f269c15da30 0x7f269c15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.108+0000 7f269affd640 1 -- 172.21.15.116:0/2348858613 >> v1:172.21.15.143:6789/0 conn(0x7f269c145230 legacy=0x7f269c15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.108+0000 7f269affd640 1 -- 172.21.15.116:0/2348858613 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c15cc40 con 0x7f269c0711c0 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.108+0000 7f269affd640 1 --2- 172.21.15.116:0/2348858613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c071650 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f269c15f3e0 tx=0x7f268402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd744e80c7ce8a68 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f269a7fc640 1 -- 172.21.15.116:0/2348858613 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684031870 con 0x7f269c0711c0 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f269a7fc640 1 -- 172.21.15.116:0/2348858613 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26840319d0 con 0x7f269c0711c0 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f269a7fc640 1 -- 172.21.15.116:0/2348858613 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684031d60 con 0x7f269c0711c0 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 -- 172.21.15.116:0/2348858613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 msgr2=0x7f269c071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 --2- 172.21.15.116:0/2348858613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c071650 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f269c15f3e0 tx=0x7f268402b910 comp rx=0 tx=0).stop 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 -- 172.21.15.116:0/2348858613 shutdown_connections 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 --2- 172.21.15.116:0/2348858613 >> v2:172.21.15.143:3300/0 conn(0x7f269c15da30 0x7f269c15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 --2- 172.21.15.116:0/2348858613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c071650 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 -- 172.21.15.116:0/2348858613 >> 172.21.15.116:0/2348858613 conn(0x7f269c067000 msgr2=0x7f269c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 -- 172.21.15.116:0/2348858613 shutdown_connections 2024-02-18T11:48:55.109 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.109+0000 7f26a218c640 1 -- 172.21.15.116:0/2348858613 wait complete. 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.110+0000 7f26a218c640 1 Processor -- start 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 -- start start 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f269c145230 0x7f269c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f269c15da30 0x7f269c1423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0c4bc0 con 0x7f269c0711c0 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0aa820 con 0x7f269c145230 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f26a218c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0aab00 con 0x7f269c15da30 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f269b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c0c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42044/0 (socket says 172.21.15.116:42044) 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f269b7fe640 1 -- 172.21.15.116:0/3195389860 learned_addr learned my addr 172.21.15.116:0/3195389860 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:55.110 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f269bfff640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f269c15da30 0x7f269c1423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.111+0000 7f269affd640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f269c145230 0x7f269c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f269b7fe640 1 -- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f269c15da30 msgr2=0x7f269c1423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f269b7fe640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f269c15da30 0x7f269c1423c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f269b7fe640 1 -- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f269c145230 msgr2=0x7f269c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f269b7fe640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f269c145230 0x7f269c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f269b7fe640 1 -- 172.21.15.116:0/3195389860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26840094d0 con 0x7f269c0711c0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f269b7fe640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c0c4680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c430 tx=0x7f268c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c011790 con 0x7f269c0711c0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c0118f0 con 0x7f269c0711c0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c011ce0 con 0x7f269c0711c0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c0aade0 con 0x7f269c0711c0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.112+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c0ab3a0 con 0x7f269c0711c0 2024-02-18T11:48:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.113+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c074520 con 0x7f269c0711c0 2024-02-18T11:48:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.113+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f268c01c460 con 0x7f269c0711c0 2024-02-18T11:48:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.114+0000 7f2693fff640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f267c06bca0 0x7f267c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.114+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f268c08a850 con 0x7f269c0711c0 2024-02-18T11:48:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.114+0000 7f269affd640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f267c06bca0 0x7f267c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.114+0000 7f269affd640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f267c06bca0 0x7f267c06e160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2684036000 tx=0x7f268403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.118+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f268c054f30 con 0x7f269c0711c0 2024-02-18T11:48:55.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.359+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f269c1438d0 con 0x7f269c0711c0 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.361+0000 7f2693fff640 1 -- 172.21.15.116:0/3195389860 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f268c0585f0 con 0x7f269c0711c0 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAc7tFlVhFMMxAAcC+CBcaJjyYSC7rVBwyvOw== 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow" 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow r" 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-02-18T11:48:55.354 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2024-02-18T11:48:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f267c06bca0 msgr2=0x7f267c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f267c06bca0 0x7f267c06e160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2684036000 tx=0x7f268403c040 comp rx=0 tx=0).stop 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 msgr2=0x7f269c0c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c0c4680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c430 tx=0x7f268c00c8f0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 shutdown_connections 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f267c06bca0 0x7f267c06e160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f269c15da30 0x7f269c1423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f269c145230 0x7f269c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 --2- 172.21.15.116:0/3195389860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c0711c0 0x7f269c0c4680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 >> 172.21.15.116:0/3195389860 conn(0x7f269c067000 msgr2=0x7f269c0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 shutdown_connections 2024-02-18T11:48:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.363+0000 7f26a218c640 1 -- 172.21.15.116:0/3195389860 wait complete. 2024-02-18T11:48:55.367 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-18T11:48:55.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.474+0000 7f318ab0c640 1 Processor -- start 2024-02-18T11:48:55.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.474+0000 7f318ab0c640 1 -- start start 2024-02-18T11:48:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f318ab0c640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f318406ea20 0x7f31841653a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.471 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f318ab0c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840b02e0 con 0x7f318415a860 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f318ab0c640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840af6d0 con 0x7f318415ac40 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f318ab0c640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31840b1780 con 0x7f318406ea20 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31888a8640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f318415a860 0x7f31840cccb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34776/0 (socket says 172.21.15.116:34776) 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31888a8640 1 -- 172.21.15.116:0/3962265066 learned_addr learned my addr 172.21.15.116:0/3962265066 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31890a9640 1 --2- 172.21.15.116:0/3962265066 >> v2:172.21.15.143:3300/0 conn(0x7f318406ea20 0x7f31841653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515313463 0 0) 0x7f31840b02e0 con 0x7f318415a860 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3174003520 con 0x7f318415a860 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2311192276 0 0) 0x7f3174003520 con 0x7f318415a860 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 >> v2:172.21.15.143:3300/0 conn(0x7f318406ea20 msgr2=0x7f31841653a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 --2- 172.21.15.116:0/3962265066 >> v2:172.21.15.143:3300/0 conn(0x7f318406ea20 0x7f31841653a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 >> v1:172.21.15.143:6789/0 conn(0x7f318415ac40 legacy=0x7f318406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840af9b0 con 0x7f318415a860 2024-02-18T11:48:55.472 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f3178001670 con 0x7f318415a860 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 >> v1:172.21.15.116:6789/0 conn(0x7f318415a860 legacy=0x7f31840cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3174003970 0x7f3174003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31740047c0 0x7f3174004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174005180 0x7f317400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840b1780 con 0x7f3174003970 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317400c020 con 0x7f31740047c0 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317400e3f0 con 0x7f3174005180 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f31888a8640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3174003970 0x7f3174003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.475+0000 7f3183fff640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31740047c0 0x7f3174004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31890a9640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174005180 0x7f317400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31888a8640 1 -- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174005180 msgr2=0x7f317400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31888a8640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174005180 0x7f317400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31888a8640 1 -- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31740047c0 msgr2=0x7f3174004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:55.473 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31888a8640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31740047c0 0x7f3174004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31888a8640 1 -- 172.21.15.116:0/3962265066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3174003520 con 0x7f3174003970 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31890a9640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174005180 0x7f317400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31890a9640 1 -- 172.21.15.116:0/3962265066 reap_dead start 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.476+0000 7f31888a8640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3174003970 0x7f3174003d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f317800c240 tx=0x7f31740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4f518691a7bdcb server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178005ad0 con 0x7f3174003970 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178005c30 con 0x7f3174003970 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f31837fe640 1 -- 172.21.15.116:0/3962265066 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317802ca90 con 0x7f3174003970 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 -- 172.21.15.116:0/3962265066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3174003970 msgr2=0x7f3174003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.474 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3174003970 0x7f3174003d90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f317800c240 tx=0x7f31740057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 -- 172.21.15.116:0/3962265066 shutdown_connections 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3962265066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3174003970 0x7f3174003d90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 -- 172.21.15.116:0/3962265066 >> 172.21.15.116:0/3962265066 conn(0x7f3184066f20 msgr2=0x7f318406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 -- 172.21.15.116:0/3962265066 shutdown_connections 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.477+0000 7f318ab0c640 1 -- 172.21.15.116:0/3962265066 wait complete. 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.478+0000 7f318ab0c640 1 Processor -- start 2024-02-18T11:48:55.475 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.478+0000 7f318ab0c640 1 -- start start 2024-02-18T11:48:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f318ab0c640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174003970 0x7f3184148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f318ab0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 0x7f31841372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f318ab0c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3184137820 0x7f31841352b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f318ab0c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184148590 con 0x7f3184134960 2024-02-18T11:48:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f318ab0c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184137cd0 con 0x7f3184137820 2024-02-18T11:48:55.478 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f318ab0c640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3184146fb0 con 0x7f3174003970 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 0x7f31841372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 0x7f31841372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42068/0 (socket says 172.21.15.116:42068) 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 -- 172.21.15.116:0/3709497628 learned_addr learned my addr 172.21.15.116:0/3709497628 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f31890a9640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3184137820 0x7f31841352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f31888a8640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174003970 0x7f3184148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 -- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174003970 msgr2=0x7f3184148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174003970 0x7f3184148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 -- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3184137820 msgr2=0x7f31841352b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3184137820 0x7f31841352b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.479+0000 7f3183fff640 1 -- 172.21.15.116:0/3709497628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184147290 con 0x7f3184134960 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f31890a9640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3184137820 0x7f31841352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f31888a8640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174003970 0x7f3184148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f3183fff640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 0x7f31841372e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f317000f5c0 tx=0x7f317000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:55.479 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700052e0 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3170005440 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184147570 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3170003830 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.480+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3184147b30 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.481+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3184147cf0 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.481+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f3170003990 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.482+0000 7f31817fa640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f316806bc70 0x7f316806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.482+0000 7f31888a8640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f316806bc70 0x7f316806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.482+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f317008a7a0 con 0x7f3184134960 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.482+0000 7f31888a8640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f316806bc70 0x7f316806e130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3178034000 tx=0x7f3178009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:55.480 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.486+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3170054df0 con 0x7f3184134960 2024-02-18T11:48:55.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.694+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3184074520 con 0x7f3184134960 2024-02-18T11:48:55.688 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.694+0000 7f31817fa640 1 -- 172.21.15.116:0/3709497628 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f31700584b0 con 0x7f3184134960 2024-02-18T11:48:55.688 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:55.688 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/673479819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6833","nonce":673479819},{"type":"v1","addr":"172.21.15.143:6835","nonce":673479819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4787,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6834/3865796101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3865796101},{"type":"v1","addr":"172.21.15.143:6834","nonce":3865796101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4987,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/4242183425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6835","nonce":4242183425},{"type":"v1","addr":"172.21.15.116:6837","nonce":4242183425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-18T11:48:46.634911+0000","modified":"2024-02-18T11:48:48.649669+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5358},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5358":{"gid":5358,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6836/1597642942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1597642942},{"type":"v1","addr":"172.21.15.116:6836","nonce":1597642942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-02-18T11:48:55.688 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 15 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f316806bc70 msgr2=0x7f316806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f316806bc70 0x7f316806e130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3178034000 tx=0x7f3178009030 comp rx=0 tx=0).stop 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 msgr2=0x7f31841372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 0x7f31841372e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f317000f5c0 tx=0x7f317000fa80 comp rx=0 tx=0).stop 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 shutdown_connections 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f316806bc70 0x7f316806e130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3184137820 0x7f31841352b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184134960 0x7f31841372e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 --2- 172.21.15.116:0/3709497628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3174003970 0x7f3184148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 >> 172.21.15.116:0/3709497628 conn(0x7f3184066f20 msgr2=0x7f3184154410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 shutdown_connections 2024-02-18T11:48:55.690 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:55.696+0000 7f318ab0c640 1 -- 172.21.15.116:0/3709497628 wait complete. 2024-02-18T11:48:56.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-18T11:48:46.634911+0000', 'modified': '2024-02-18T11:48:48.649669+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5358}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5358': {'gid': 5358, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6836/1597642942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 1597642942}, {'type': 'v1', 'addr': '172.21.15.116:6836', 'nonce': 1597642942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-18T11:48:56.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-18T11:48:56.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-18T11:48:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.212+0000 7f893398a640 1 Processor -- start 2024-02-18T11:48:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.212+0000 7f893398a640 1 -- start start 2024-02-18T11:48:56.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.213+0000 7f893398a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c072160 0x7f892c164e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.213+0000 7f893398a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c15a480 con 0x7f892c150ed0 2024-02-18T11:48:56.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.213+0000 7f893398a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c15a760 con 0x7f892c0cdd20 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.213+0000 7f893398a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c15aa40 con 0x7f892c072160 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.213+0000 7f8931726640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f892c150ed0 0x7f892c0b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34790/0 (socket says 172.21.15.116:34790) 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.213+0000 7f8931726640 1 -- 172.21.15.116:0/1306813196 learned_addr learned my addr 172.21.15.116:0/1306813196 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8931f27640 1 --2- 172.21.15.116:0/1306813196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c072160 0x7f892c164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654987943 0 0) 0x7f892c15a480 con 0x7f892c150ed0 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8918003520 con 0x7f892c150ed0 2024-02-18T11:48:56.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936937371 0 0) 0x7f892c15a760 con 0x7f892c0cdd20 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f892c15a480 con 0x7f892c0cdd20 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1902484404 0 0) 0x7f8918003520 con 0x7f892c150ed0 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8931f27640 1 -- 172.21.15.116:0/1306813196 >> v1:172.21.15.143:6789/0 conn(0x7f892c0cdd20 legacy=0x7f892c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8931f27640 1 -- 172.21.15.116:0/1306813196 >> v1:172.21.15.116:6789/0 conn(0x7f892c150ed0 legacy=0x7f892c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.214+0000 7f8931f27640 1 -- 172.21.15.116:0/1306813196 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c15ad20 con 0x7f892c072160 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.215+0000 7f8931f27640 1 --2- 172.21.15.116:0/1306813196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c072160 0x7f892c164e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f892400af00 tx=0x7f892402bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acd5f62372d050f server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.215+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c001670 con 0x7f892c072160 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.215+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892400a2e0 con 0x7f892c072160 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.215+0000 7f8923fff640 1 -- 172.21.15.116:0/1306813196 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892402c880 con 0x7f892c072160 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.215+0000 7f893398a640 1 -- 172.21.15.116:0/1306813196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c072160 msgr2=0x7f892c164e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.215+0000 7f893398a640 1 --2- 172.21.15.116:0/1306813196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c072160 0x7f892c164e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f892400af00 tx=0x7f892402bd60 comp rx=0 tx=0).stop 2024-02-18T11:48:56.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.216+0000 7f893398a640 1 -- 172.21.15.116:0/1306813196 shutdown_connections 2024-02-18T11:48:56.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.216+0000 7f893398a640 1 --2- 172.21.15.116:0/1306813196 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c072160 0x7f892c164e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.216+0000 7f893398a640 1 -- 172.21.15.116:0/1306813196 >> 172.21.15.116:0/1306813196 conn(0x7f892c067000 msgr2=0x7f892c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.216+0000 7f893398a640 1 -- 172.21.15.116:0/1306813196 shutdown_connections 2024-02-18T11:48:56.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.216+0000 7f893398a640 1 -- 172.21.15.116:0/1306813196 wait complete. 2024-02-18T11:48:56.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 Processor -- start 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 -- start start 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f892c072160 0x7f892c15d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c0cdd20 0x7f892c1495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 0x7f892c14fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c1ef650 con 0x7f892c150ed0 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0a9930 con 0x7f892c072160 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.217+0000 7f893398a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c15e270 con 0x7f892c0cdd20 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 0x7f892c14fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 0x7f892c14fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42078/0 (socket says 172.21.15.116:42078) 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 -- 172.21.15.116:0/1692101806 learned_addr learned my addr 172.21.15.116:0/1692101806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931726640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f892c072160 0x7f892c15d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8930f25640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c0cdd20 0x7f892c1495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 -- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c0cdd20 msgr2=0x7f892c1495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.222 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c0cdd20 0x7f892c1495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 -- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f892c072160 msgr2=0x7f892c15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f892c072160 0x7f892c15d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 -- 172.21.15.116:0/1692101806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892400ab80 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931726640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f892c072160 0x7f892c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8930f25640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c0cdd20 0x7f892c1495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8931f27640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 0x7f892c14fad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8924036040 tx=0x7f8924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8921ffb640 1 -- 172.21.15.116:0/1692101806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892402dcd0 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8921ffb640 1 -- 172.21.15.116:0/1692101806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892402de30 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c0a88f0 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f8921ffb640 1 -- 172.21.15.116:0/1692101806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924004280 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.218+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c0a9130 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.220+0000 7f8921ffb640 1 -- 172.21.15.116:0/1692101806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f89240043e0 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.220+0000 7f8921ffb640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f891006bc70 0x7f891006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.221+0000 7f8921ffb640 1 -- 172.21.15.116:0/1692101806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89240ad0d0 con 0x7f892c150ed0 2024-02-18T11:48:56.223 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.221+0000 7f8931726640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f891006bc70 0x7f891006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.221+0000 7f8931726640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f891006bc70 0x7f891006e130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f891c001340 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.221+0000 7f8936638740 1 Processor -- start 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.221+0000 7f8936638740 1 -- start start 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8936638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 0x55ee937c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937c64e0 0x55ee9379cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8936638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee93759dc0 con 0x55ee937e48f0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9370cf20 con 0x55ee937e6dd0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee9370a450 con 0x55ee937c64e0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 0x55ee937c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 0x55ee937c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42090/0 (socket says 172.21.15.116:42090) 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 -- 172.21.15.116:0/2925532717 learned_addr learned my addr 172.21.15.116:0/2925532717 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8931726640 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.224 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8931f27640 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937c64e0 0x55ee9379cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 -- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937c64e0 msgr2=0x55ee9379cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937c64e0 0x55ee9379cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 -- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 msgr2=0x55ee937c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8930f25640 1 -- 172.21.15.116:0/2925532717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee936baf20 con 0x55ee937e48f0 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.222+0000 7f8931f27640 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937c64e0 0x55ee9379cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8930f25640 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 0x55ee937c93f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f89280015b0 tx=0x7f8928001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5044a476540186b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8916ffd640 1 -- 172.21.15.116:0/2925532717 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800d930 con 0x55ee937e48f0 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8916ffd640 1 -- 172.21.15.116:0/2925532717 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800da90 con 0x55ee937e48f0 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8916ffd640 1 -- 172.21.15.116:0/2925532717 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800de00 con 0x55ee937e48f0 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 -- 172.21.15.116:0/2925532717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 msgr2=0x55ee937c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 0x55ee937c93f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f89280015b0 tx=0x7f8928001a70 comp rx=0 tx=0).stop 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 -- 172.21.15.116:0/2925532717 shutdown_connections 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937c64e0 0x55ee9379cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.225 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937e48f0 0x55ee937c93f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 --2- 172.21.15.116:0/2925532717 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 -- 172.21.15.116:0/2925532717 >> 172.21.15.116:0/2925532717 conn(0x55ee9379b330 msgr2=0x55ee937e3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 -- 172.21.15.116:0/2925532717 shutdown_connections 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.223+0000 7f8936638740 1 -- 172.21.15.116:0/2925532717 wait complete. 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.224+0000 7f8936638740 1 -- start start 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.224+0000 7f8936638740 1 Processor -- start 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee937f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee93705c30 con 0x55ee937f83f0 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee936aad30 con 0x55ee937e6dd0 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee933d9310 con 0x55ee937e48f0 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8936638740 10 client.0 _ll_register_callbacks cb 0x55ee936b2d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee937f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.226 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee937f8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42102/0 (socket says 172.21.15.116:42102) 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 -- 172.21.15.116:0/4148205497 learned_addr learned my addr 172.21.15.116:0/4148205497 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8930f25640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931726640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 -- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 msgr2=0x55ee937caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 -- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 msgr2=0x55ee937c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 -- 172.21.15.116:0/4148205497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c15a480 con 0x55ee937f83f0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f8931f27640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee937f8870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8924077a00 tx=0x7f892407bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c784a0bc8c1406 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.225+0000 7f890bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f890bfff640 1 -- 172.21.15.116:0/4148205497 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924093170 con 0x55ee937f83f0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f890bfff640 1 -- 172.21.15.116:0/4148205497 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8924083cb0 con 0x55ee937f83f0 2024-02-18T11:48:56.227 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee934056c0 con 0x55ee937f83f0 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f890bfff640 1 -- 172.21.15.116:0/4148205497 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8924004890 con 0x55ee937f83f0 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 10 client.5565 fetch_fsmap learned FSMap version 15 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55ee937042b0 con 0x55ee937f83f0 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f890bfff640 1 -- 172.21.15.116:0/4148205497 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ee934056c0 con 0x55ee937f83f0 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f890bfff640 1 -- 172.21.15.116:0/4148205497 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f892404f020 con 0x55ee937f83f0 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 10 client.5565 fetch_fsmap finished waiting for FSMap version 15 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 10 client.5565 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 20 client.5565 populate_metadata read hostname 'smithi116' 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee93803d30 0x55ee93806120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 4 client.5565 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 --> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee93456270 con 0x55ee93803d30 2024-02-18T11:48:56.228 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8931726640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee93803d30 0x55ee93806120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f890bfff640 10 client.5565 ms_handle_connect on v2:172.21.15.116:6834/1597642942 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.226+0000 7f8931726640 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee93803d30 0x55ee93806120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5153578d27e5cfc6 server_cookie=99667ac9c8774549 in_seq=0 out_seq=0 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f890bfff640 1 -- 172.21.15.116:0/4148205497 <== mds.0 v2:172.21.15.116:6834/1597642942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee93456270 con 0x55ee93803d30 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f890bfff640 10 client.5565 handle_command_reply: tid=0 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 client.5565 shutdown 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 msgr2=0x55ee937f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee937f8870 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8924077a00 tx=0x7f892407bca0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 shutdown_connections 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee93803d30 0x55ee93806120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee937f8870 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 --2- 172.21.15.116:0/4148205497 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.229 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f890bfff640 0 client.5565 ms_handle_reset on v2:172.21.15.116:6834/1597642942 2024-02-18T11:48:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 >> 172.21.15.116:0/4148205497 conn(0x55ee937c64e0 msgr2=0x55ee9379d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 shutdown_connections 2024-02-18T11:48:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 1 -- 172.21.15.116:0/4148205497 wait complete. 2024-02-18T11:48:56.230 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.227+0000 7f8936638740 20 client.5565 trim_cache size 0 max 16384 2024-02-18T11:48:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.243+0000 7f8936638740 1 Processor -- start 2024-02-18T11:48:56.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.243+0000 7f8936638740 1 -- start start 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee9379d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8936638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee9380a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8936638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9368b490 con 0x55ee937f83f0 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee937982f0 con 0x55ee937e6dd0 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee937e4e40 con 0x55ee937e48f0 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee9380a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee9380a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42114/0 (socket says 172.21.15.116:42114) 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 -- 172.21.15.116:0/3774874912 learned_addr learned my addr 172.21.15.116:0/3774874912 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:56.244 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8930f25640 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931726640 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee9379d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 -- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 msgr2=0x55ee9379d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee9379d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 -- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 msgr2=0x55ee937ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 -- 172.21.15.116:0/3774874912 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee936a0a20 con 0x55ee937f83f0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931726640 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee9379d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f8931f27640 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee9380a140 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f892407b760 tx=0x7f892407d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c737cffb2a7a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f89167fc640 1 -- 172.21.15.116:0/3774874912 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892407af00 con 0x55ee937f83f0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f89167fc640 1 -- 172.21.15.116:0/3774874912 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8924077840 con 0x55ee937f83f0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.244+0000 7f89167fc640 1 -- 172.21.15.116:0/3774874912 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924083cb0 con 0x55ee937f83f0 2024-02-18T11:48:56.245 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 -- 172.21.15.116:0/3774874912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 msgr2=0x55ee9380a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee9380a140 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f892407b760 tx=0x7f892407d000 comp rx=0 tx=0).stop 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 -- 172.21.15.116:0/3774874912 shutdown_connections 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee9380a140 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee937ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 --2- 172.21.15.116:0/3774874912 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee9379d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 -- 172.21.15.116:0/3774874912 >> 172.21.15.116:0/3774874912 conn(0x55ee937c64e0 msgr2=0x55ee937f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 -- 172.21.15.116:0/3774874912 shutdown_connections 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 -- 172.21.15.116:0/3774874912 wait complete. 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 -- start start 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.245+0000 7f8936638740 1 Processor -- start 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee93856ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee938570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9379ac40 con 0x55ee937f83f0 2024-02-18T11:48:56.246 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9379d480 con 0x55ee937e6dd0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee937f7be0 con 0x55ee937e48f0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8936638740 10 client.0 _ll_register_callbacks cb 0x7f8928007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee938570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8931f27640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee938570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42118/0 (socket says 172.21.15.116:42118) 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8931f27640 1 -- 172.21.15.116:0/2277285806 learned_addr learned my addr 172.21.15.116:0/2277285806 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8930f25640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee93856ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.246+0000 7f8931726640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931f27640 1 -- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 msgr2=0x55ee937cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931f27640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931f27640 1 -- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 msgr2=0x55ee93856ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931f27640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee93856ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931f27640 1 -- 172.21.15.116:0/2277285806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee937042b0 con 0x55ee937f83f0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931726640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931f27640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee938570e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f892402bda0 tx=0x7f89240ab340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43251d7b670ab00 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.247 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f890bfff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f890bfff640 1 -- 172.21.15.116:0/2277285806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892407af00 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f890bfff640 1 -- 172.21.15.116:0/2277285806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f892408f2e0 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f892800da90 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f890bfff640 1 -- 172.21.15.116:0/2277285806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8924083cb0 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 10 client.5571 fetch_fsmap learned FSMap version 15 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55ee936c6780 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f890bfff640 1 -- 172.21.15.116:0/2277285806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800da90 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f890bfff640 1 -- 172.21.15.116:0/2277285806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f892403c070 con 0x55ee937f83f0 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 10 client.5571 fetch_fsmap finished waiting for FSMap version 15 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 10 client.5571 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 20 client.5571 populate_metadata read hostname 'smithi116' 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee9380a2f0 0x55ee9380a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 4 client.5571 mds_command: new command op to 5358 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-18T11:48:56.248 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 --> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ee93442610 con 0x55ee9380a2f0 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.247+0000 7f8931726640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee9380a2f0 0x55ee9380a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f890bfff640 10 client.5571 ms_handle_connect on v2:172.21.15.116:6834/1597642942 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8931726640 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee9380a2f0 0x55ee9380a6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32580d6d9c7b60a server_cookie=90e2c93b6e89fddc in_seq=0 out_seq=0 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f890bfff640 1 -- 172.21.15.116:0/2277285806 <== mds.0 v2:172.21.15.116:6834/1597642942 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ee93442610 con 0x55ee9380a2f0 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f890bfff640 10 client.5571 handle_command_reply: tid=0 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 client.5571 shutdown 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 msgr2=0x55ee938570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee938570e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f892402bda0 tx=0x7f89240ab340 comp rx=0 tx=0).stop 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 shutdown_connections 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:6834/1597642942,v1:172.21.15.116:6836/1597642942] conn(0x55ee9380a2f0 0x55ee9380a6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ee937f83f0 0x55ee938570e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f890bfff640 0 client.5571 ms_handle_reset on v2:172.21.15.116:6834/1597642942 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ee937e6dd0 0x55ee93856ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.249 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 --2- 172.21.15.116:0/2277285806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x55ee937e48f0 0x55ee937cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 >> 172.21.15.116:0/2277285806 conn(0x55ee937c64e0 msgr2=0x55ee937c43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 shutdown_connections 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 1 -- 172.21.15.116:0/2277285806 wait complete. 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.248+0000 7f8936638740 20 client.5571 trim_cache size 0 max 16384 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_fsid": "82e3e917-bb19-48fa-a66a-62df77a0f413", 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout: "whoami": 0, 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 5358, 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout: "want_state": "up:active", 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout: "state": "up:active", 2024-02-18T11:48:56.250 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs", 2024-02-18T11:48:56.251 INFO:teuthology.orchestra.run.smithi116.stdout: "rank_uptime": 8.9506580390000003, 2024-02-18T11:48:56.251 INFO:teuthology.orchestra.run.smithi116.stdout: "mdsmap_epoch": 15, 2024-02-18T11:48:56.251 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch": 38, 2024-02-18T11:48:56.251 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_epoch_barrier": 38, 2024-02-18T11:48:56.251 INFO:teuthology.orchestra.run.smithi116.stdout: "uptime": 15.93421672 2024-02-18T11:48:56.251 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-02-18T11:48:56.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f891006bc70 msgr2=0x7f891006e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f891006bc70 0x7f891006e130 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f891c001340 tx=0x7f891c009040 comp rx=0 tx=0).stop 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 msgr2=0x7f892c14fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 0x7f892c14fad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8924036040 tx=0x7f8924004a40 comp rx=0 tx=0).stop 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 shutdown_connections 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f891006bc70 0x7f891006e130 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f892c150ed0 0x7f892c14fad0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f892c0cdd20 0x7f892c1495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 --2- 172.21.15.116:0/1692101806 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f892c072160 0x7f892c15d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 >> 172.21.15.116:0/1692101806 conn(0x7f892c067000 msgr2=0x7f892c14df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 shutdown_connections 2024-02-18T11:48:56.253 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.249+0000 7f893398a640 1 -- 172.21.15.116:0/1692101806 wait complete. 2024-02-18T11:48:56.254 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-02-18T11:48:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.363+0000 7f1d07cd4640 1 Processor -- start 2024-02-18T11:48:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.363+0000 7f1d07cd4640 1 -- start start 2024-02-18T11:48:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d07cd4640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d000d1080 0x7f1d000b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.363 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d07cd4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1d000ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d07cd4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d000ce780 con 0x7f1d0015dc60 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d07cd4640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d000d3730 con 0x7f1d000d1080 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d07cd4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cf250 con 0x7f1d00161850 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d05a70640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1d0015dc60 0x7f1d000d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34802/0 (socket says 172.21.15.116:34802) 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d05a70640 1 -- 172.21.15.116:0/1046800759 learned_addr learned my addr 172.21.15.116:0/1046800759 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d06271640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1d000ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d0526f640 1 --2- 172.21.15.116:0/1046800759 >> v2:172.21.15.143:3300/0 conn(0x7f1d000d1080 0x7f1d000b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186336050 0 0) 0x7f1d000ce780 con 0x7f1d0015dc60 2024-02-18T11:48:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cec003520 con 0x7f1d0015dc60 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262517534 0 0) 0x7f1cec003520 con 0x7f1d0015dc60 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 msgr2=0x7f1d000ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1d000ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 >> v2:172.21.15.143:3300/0 conn(0x7f1d000d1080 msgr2=0x7f1d000b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 --2- 172.21.15.116:0/1046800759 >> v2:172.21.15.143:3300/0 conn(0x7f1d000d1080 0x7f1d000b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000ce9b0 con 0x7f1d0015dc60 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f1cfc001670 con 0x7f1d0015dc60 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 >> v1:172.21.15.116:6789/0 conn(0x7f1d0015dc60 legacy=0x7f1d000d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1cec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1cec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cec004d00 0x7f1cec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000d3730 con 0x7f1cec003970 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cf250 con 0x7f1cec004d00 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cec00b980 con 0x7f1d00161850 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.364+0000 7f1d05a70640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1cec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d06271640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cec004d00 0x7f1cec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d0526f640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1cec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d05a70640 1 -- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 msgr2=0x7f1cec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d05a70640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1cec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d05a70640 1 -- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cec004d00 msgr2=0x7f1cec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d05a70640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cec004d00 0x7f1cec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d05a70640 1 -- 172.21.15.116:0/1046800759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec003520 con 0x7f1cec003970 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d0526f640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d00161850 0x7f1cec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.365+0000 7f1d05a70640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1cec003d90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c240 tx=0x7f1cec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9ba4ae0a45628a server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc02cb00 con 0x7f1cec003970 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc02cc60 con 0x7f1cec003970 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d04a6e640 1 -- 172.21.15.116:0/1046800759 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc005af0 con 0x7f1cec003970 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d07cd4640 1 -- 172.21.15.116:0/1046800759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 msgr2=0x7f1cec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/1046800759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1cec003d90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c240 tx=0x7f1cec0057f0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d06271640 1 -- 172.21.15.116:0/1046800759 reap_dead start 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d07cd4640 1 -- 172.21.15.116:0/1046800759 shutdown_connections 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d07cd4640 1 -- 172.21.15.116:0/1046800759 >> 172.21.15.116:0/1046800759 conn(0x7f1d000b5910 msgr2=0x7f1d000b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d07cd4640 1 -- 172.21.15.116:0/1046800759 shutdown_connections 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.366+0000 7f1d07cd4640 1 -- 172.21.15.116:0/1046800759 wait complete. 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.367+0000 7f1d07cd4640 1 Processor -- start 2024-02-18T11:48:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.367+0000 7f1d07cd4640 1 -- start start 2024-02-18T11:48:56.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d07cd4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1d000cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d07cd4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000c9810 0x7f1d000cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d07cd4640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d000d38d0 0x7f1d000d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d07cd4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000d4290 con 0x7f1cec003970 2024-02-18T11:48:56.370 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d07cd4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cd480 con 0x7f1d000c9810 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d07cd4640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cb180 con 0x7f1d000d38d0 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1d000cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1d000cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42136/0 (socket says 172.21.15.116:42136) 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 -- 172.21.15.116:0/3502278409 learned_addr learned my addr 172.21.15.116:0/3502278409 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d0526f640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000c9810 0x7f1d000cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d06271640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d000d38d0 0x7f1d000d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 -- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d000d38d0 msgr2=0x7f1d000d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d000d38d0 0x7f1d000d3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 -- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000c9810 msgr2=0x7f1d000cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000c9810 0x7f1d000cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.368+0000 7f1d05a70640 1 -- 172.21.15.116:0/3502278409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000cd6d0 con 0x7f1cec003970 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1d0526f640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000c9810 0x7f1d000cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1d05a70640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1d000cc980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c240 tx=0x7f1cfc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.371 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc039070 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cfc004bf0 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d000cd9b0 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc03c040 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.369+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d000cc460 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.370+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d000cc620 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.370+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f1cfc032070 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.371+0000 7f1cf67fc640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1ce406bca0 0x7f1ce406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.371+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cfc0abba0 con 0x7f1cec003970 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.371+0000 7f1d0526f640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1ce406bca0 0x7f1ce406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.371+0000 7f1d0526f640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1ce406bca0 0x7f1ce406e160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d000b47c0 tx=0x7f1cf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:48:56.372 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.375+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cfc076280 con 0x7f1cec003970 2024-02-18T11:48:56.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.584+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d00067840 con 0x7f1cec003970 2024-02-18T11:48:56.577 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.584+0000 7f1cf67fc640 1 -- 172.21.15.116:0/3502278409 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1cfc079940 con 0x7f1cec003970 2024-02-18T11:48:56.577 INFO:teuthology.orchestra.run.smithi116.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-18T11:48:56.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1ce406bca0 msgr2=0x7f1ce406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.579 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1ce406bca0 0x7f1ce406e160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d000b47c0 tx=0x7f1cf0008040 comp rx=0 tx=0).stop 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 msgr2=0x7f1d000cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1d000cc980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00c240 tx=0x7f1cfc001a00 comp rx=0 tx=0).stop 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 shutdown_connections 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1ce406bca0 0x7f1ce406e160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d000d38d0 0x7f1d000d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d000c9810 0x7f1d000cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 --2- 172.21.15.116:0/3502278409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cec003970 0x7f1d000cc980 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 >> 172.21.15.116:0/3502278409 conn(0x7f1d000b5910 msgr2=0x7f1d000b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 shutdown_connections 2024-02-18T11:48:56.580 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:48:56.586+0000 7f1d07cd4640 1 -- 172.21.15.116:0/3502278409 wait complete. 2024-02-18T11:48:56.590 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-18T11:48:56.590 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-18T11:48:56.590 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-02-18T11:48:56.591 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-18T11:48:56.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:56.591 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-18T11:48:56.591 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-02-18T11:48:56.591 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-18T11:48:56.591 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:48:56.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:56.591 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:56.608 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42697sec preferred_lft 42697sec 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 26:00:e6:36:a7:7d brd ff:ff:ff:ff:ff:ff 2024-02-18T11:48:56.609 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::6c0a:43ff:fedd:aff6/64 scope link 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 8a:a8:c5:f8:ac:12 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::88a8:c5ff:fef8:ac12/64 scope link 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 26:00:e6:36:a7:7d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2400:e6ff:fe36:a77d/64 scope link 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-02-18T11:48:56.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:56.611 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-18T11:48:56.673 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-02-18T11:48:56.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:56.675 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-02-18T11:48:56.730 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-02-18T11:48:56.731 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-18T11:48:56.731 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-18T11:48:56.731 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-18T11:48:56.731 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-18T11:48:56.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:56.731 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec bash -c ' 2024-02-18T11:48:56.732 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-02-18T11:48:56.732 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-18T11:48:56.732 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-18T11:48:56.732 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-18T11:48:56.732 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-02-18T11:48:56.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:56.827 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-02-18T11:48:56.842 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-02-18T11:48:56.843 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-18T11:48:56.843 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:48:56.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:56.843 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-02-18T11:48:56.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:56.910 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:56.964 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-18T11:48:56.965 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-18T11:48:56.965 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:57.020 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-02-18T11:48:57.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:57.021 DEBUG:teuthology.orchestra.run.smithi116:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:57.104 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:57.104 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-18T11:48:57.104 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:57.113 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:57.113 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-18T11:48:57.113 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:57.113 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:57.113 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:invalid new device string format 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:3300,172.21.15.143:3300,172.21.15.143:3301" 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: switching to using v1 address with old syntax 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: resolved to: "172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790" 2024-02-18T11:48:57.120 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.116:6789,172.21.15.143:6789,172.21.15.143:6790:/ 2024-02-18T11:48:57.121 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413" will pass to kernel 2024-02-18T11:48:57.121 INFO:teuthology.orchestra.run.smithi116.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-02-18T11:48:57.121 INFO:teuthology.orchestra.run.smithi116.stdout: You just mounted a file system that supports labels which does not 2024-02-18T11:48:57.121 INFO:teuthology.orchestra.run.smithi116.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-02-18T11:48:57.121 INFO:teuthology.orchestra.run.smithi116.stdout: applications will generate AVC messages and not be allowed access to 2024-02-18T11:48:57.121 INFO:teuthology.orchestra.run.smithi116.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-02-18T11:48:57.123 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-18T11:48:57.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:57.123 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-18T11:48:57.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:57.190 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-18T11:48:57.250 INFO:teuthology.orchestra.run.smithi116.stdout:client4886 2024-02-18T11:48:57.250 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:48:57.250 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T11:48:57.306 DEBUG:teuthology.orchestra.run.smithi116:> mktemp -d --suffix=xfstests-dev 2024-02-18T11:48:57.361 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.hC7pXm5hUExfstests-dev 2024-02-18T11:48:57.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:57.361 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.hC7pXm5hUExfstests-dev) 2024-02-18T11:48:57.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:57.429 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.hC7pXm5hUExfstests-dev) 2024-02-18T11:48:57.487 INFO:teuthology.orchestra.run.smithi116.stderr:Cloning into '/tmp/tmp.hC7pXm5hUExfstests-dev'... 2024-02-18T11:48:58.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:58.179 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-02-18T11:48:58.239 DEBUG:teuthology.orchestra.run.smithi116:> mktemp -d --suffix=test 2024-02-18T11:48:58.294 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.jguMaGqYX4test 2024-02-18T11:48:58.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:48:58.294 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-02-18T11:48:58.350 DEBUG:teuthology.orchestra.run.smithi116:> mktemp -d --suffix=scratch 2024-02-18T11:48:58.408 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:48:58.408 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-02-18T11:48:58.409 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-02-18T11:48:59.033 INFO:teuthology.orchestra.run.smithi116.stdout:Last metadata expiration check: 0:03:44 ago on Sun 18 Feb 2024 11:45:15 AM UTC. 2024-02-18T11:48:59.147 INFO:teuthology.orchestra.run.smithi116.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-02-18T11:48:59.148 INFO:teuthology.orchestra.run.smithi116.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-02-18T11:48:59.149 INFO:teuthology.orchestra.run.smithi116.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-02-18T11:48:59.150 INFO:teuthology.orchestra.run.smithi116.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-02-18T11:48:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-02-18T11:48:59.153 INFO:teuthology.orchestra.run.smithi116.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-02-18T11:48:59.155 INFO:teuthology.orchestra.run.smithi116.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-02-18T11:48:59.156 INFO:teuthology.orchestra.run.smithi116.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-02-18T11:48:59.158 INFO:teuthology.orchestra.run.smithi116.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-02-18T11:48:59.161 INFO:teuthology.orchestra.run.smithi116.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-02-18T11:48:59.162 INFO:teuthology.orchestra.run.smithi116.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-02-18T11:48:59.164 INFO:teuthology.orchestra.run.smithi116.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-02-18T11:48:59.165 INFO:teuthology.orchestra.run.smithi116.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-02-18T11:48:59.166 INFO:teuthology.orchestra.run.smithi116.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-02-18T11:48:59.167 INFO:teuthology.orchestra.run.smithi116.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-02-18T11:48:59.168 INFO:teuthology.orchestra.run.smithi116.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-02-18T11:48:59.169 INFO:teuthology.orchestra.run.smithi116.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-02-18T11:48:59.170 INFO:teuthology.orchestra.run.smithi116.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-02-18T11:48:59.171 INFO:teuthology.orchestra.run.smithi116.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-02-18T11:48:59.174 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-02-18T11:48:59.238 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: kernel-devel x86_64 5.14.0-421.el9 CentOS-AppStream 22 M 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2024-02-18T11:48:59.242 INFO:teuthology.orchestra.run.smithi116.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrading: 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: cpp x86_64 11.4.1-3.el9 CentOS-AppStream 11 M 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper x86_64 9:1.02.197-2.el9 CentOS-BaseOS 139 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 CentOS-BaseOS 33 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 CentOS-BaseOS 31 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 CentOS-BaseOS 178 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: e2fsprogs x86_64 1.46.5-5.el9 CentOS-BaseOS 1.0 M 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 CentOS-BaseOS 221 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: gcc x86_64 11.4.1-3.el9 CentOS-AppStream 32 M 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: gcc-c++ x86_64 11.4.1-3.el9 CentOS-AppStream 13 M 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 CentOS-AppStream 46 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: git x86_64 2.43.0-1.el9 CentOS-AppStream 50 k 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: git-core x86_64 2.43.0-1.el9 CentOS-AppStream 4.4 M 2024-02-18T11:48:59.243 INFO:teuthology.orchestra.run.smithi116.stdout: git-core-doc noarch 2.43.0-1.el9 CentOS-AppStream 2.6 M 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: libcom_err x86_64 1.46.5-5.el9 CentOS-BaseOS 25 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: libgcc x86_64 11.4.1-3.el9 CentOS-BaseOS 91 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: libgomp x86_64 11.4.1-3.el9 CentOS-BaseOS 267 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: libss x86_64 1.46.5-5.el9 CentOS-BaseOS 31 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: libstdc++ x86_64 11.4.1-3.el9 CentOS-BaseOS 739 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: libstdc++-devel x86_64 11.4.1-3.el9 CentOS-AppStream 2.2 M 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: lvm2 x86_64 9:2.03.23-2.el9 CentOS-BaseOS 1.5 M 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 CentOS-BaseOS 1.0 M 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: make x86_64 1:4.3-8.el9 CentOS-BaseOS 529 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: perl-Git noarch 2.43.0-1.el9 CentOS-AppStream 37 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-02-18T11:48:59.244 INFO:teuthology.orchestra.run.smithi116.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: libnbd x86_64 1.18.1-3.el9 CentOS-AppStream 165 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout:Installing weak dependencies: 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-02-18T11:48:59.245 INFO:teuthology.orchestra.run.smithi116.stdout:================================================================================ 2024-02-18T11:48:59.246 INFO:teuthology.orchestra.run.smithi116.stdout:Install 17 Packages 2024-02-18T11:48:59.246 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrade 28 Packages 2024-02-18T11:48:59.246 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:48:59.246 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 108 M 2024-02-18T11:48:59.246 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-02-18T11:48:59.803 INFO:teuthology.orchestra.run.smithi116.stdout:(1/45): fio-engine-libaio-3.35-1.el9.x86_64.rpm 64 kB/s | 14 kB 00:00 2024-02-18T11:48:59.828 INFO:teuthology.orchestra.run.smithi116.stdout:(2/45): fio-engine-http-3.35-1.el9.x86_64.rpm 68 kB/s | 17 kB 00:00 2024-02-18T11:48:59.986 INFO:teuthology.orchestra.run.smithi116.stdout:(3/45): fio-engine-nbd-3.35-1.el9.x86_64.rpm 74 kB/s | 13 kB 00:00 2024-02-18T11:49:00.020 INFO:teuthology.orchestra.run.smithi116.stdout:(4/45): fio-engine-rados-3.35-1.el9.x86_64.rpm 76 kB/s | 15 kB 00:00 2024-02-18T11:49:00.137 INFO:teuthology.orchestra.run.smithi116.stdout:(5/45): fio-engine-rbd-3.35-1.el9.x86_64.rpm 100 kB/s | 15 kB 00:00 2024-02-18T11:49:00.162 INFO:teuthology.orchestra.run.smithi116.stdout:(6/45): fio-engine-rdma-3.35-1.el9.x86_64.rpm 139 kB/s | 20 kB 00:00 2024-02-18T11:49:00.238 INFO:teuthology.orchestra.run.smithi116.stdout:(7/45): fio-3.35-1.el9.x86_64.rpm 9.4 MB/s | 6.2 MB 00:00 2024-02-18T11:49:00.296 INFO:teuthology.orchestra.run.smithi116.stdout:(8/45): libcap-devel-2.48-9.el9.x86_64.rpm 248 kB/s | 33 kB 00:00 2024-02-18T11:49:00.422 INFO:teuthology.orchestra.run.smithi116.stdout:(9/45): libnbd-1.18.1-3.el9.x86_64.rpm 909 kB/s | 165 kB 00:00 2024-02-18T11:49:00.680 INFO:teuthology.orchestra.run.smithi116.stdout:(10/45): kernel-devel-5.14.0-421.el9.x86_64.rpm 40 MB/s | 22 MB 00:00 2024-02-18T11:49:00.714 INFO:teuthology.orchestra.run.smithi116.stdout:(11/45): sqlite-3.34.1-7.el9.x86_64.rpm 1.8 MB/s | 747 kB 00:00 2024-02-18T11:49:00.739 INFO:teuthology.orchestra.run.smithi116.stdout:(12/45): udftools-2.2-5.el9.x86_64.rpm 457 kB/s | 144 kB 00:00 2024-02-18T11:49:00.764 INFO:teuthology.orchestra.run.smithi116.stdout:(13/45): rmt-1.6-6.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-02-18T11:49:00.906 INFO:teuthology.orchestra.run.smithi116.stdout:(14/45): gdbm-devel-1.19-4.el9.x86_64.rpm 351 kB/s | 59 kB 00:00 2024-02-18T11:49:00.957 INFO:teuthology.orchestra.run.smithi116.stdout:(15/45): gdbm-1.19-4.el9.x86_64.rpm 546 kB/s | 132 kB 00:00 2024-02-18T11:49:01.065 INFO:teuthology.orchestra.run.smithi116.stdout:(16/45): dump-0.4-0.51.b47.el9.x86_64.rpm 292 kB/s | 88 kB 00:00 2024-02-18T11:49:01.191 INFO:teuthology.orchestra.run.smithi116.stdout:(17/45): cpp-11.4.1-3.el9.x86_64.rpm 51 MB/s | 11 MB 00:00 2024-02-18T11:49:01.283 INFO:teuthology.orchestra.run.smithi116.stdout:(18/45): indent-2.2.13-5.el9.x86_64.rpm 479 kB/s | 179 kB 00:00 2024-02-18T11:49:01.358 INFO:teuthology.orchestra.run.smithi116.stdout:(19/45): gcc-plugin-annobin-11.4.1-3.el9.x86_64 632 kB/s | 46 kB 00:00 2024-02-18T11:49:01.483 INFO:teuthology.orchestra.run.smithi116.stdout:(20/45): gcc-c++-11.4.1-3.el9.x86_64.rpm 44 MB/s | 13 MB 00:00 2024-02-18T11:49:01.800 INFO:teuthology.orchestra.run.smithi116.stdout:(21/45): gcc-11.4.1-3.el9.x86_64.rpm 44 MB/s | 32 MB 00:00 2024-02-18T11:49:01.825 INFO:teuthology.orchestra.run.smithi116.stdout:(22/45): git-2.43.0-1.el9.x86_64.rpm 106 kB/s | 50 kB 00:00 2024-02-18T11:49:01.892 INFO:teuthology.orchestra.run.smithi116.stdout:(23/45): git-core-2.43.0-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-02-18T11:49:01.942 INFO:teuthology.orchestra.run.smithi116.stdout:(24/45): libstdc++-devel-11.4.1-3.el9.x86_64.rp 19 MB/s | 2.2 MB 00:00 2024-02-18T11:49:02.001 INFO:teuthology.orchestra.run.smithi116.stdout:(25/45): git-core-doc-2.43.0-1.el9.noarch.rpm 15 MB/s | 2.6 MB 00:00 2024-02-18T11:49:02.026 INFO:teuthology.orchestra.run.smithi116.stdout:(26/45): perl-Git-2.43.0-1.el9.noarch.rpm 276 kB/s | 37 kB 00:00 2024-02-18T11:49:02.103 INFO:teuthology.orchestra.run.smithi116.stdout:(27/45): python-unversioned-command-3.9.18-3.el 52 kB/s | 8.3 kB 00:00 2024-02-18T11:49:02.127 INFO:teuthology.orchestra.run.smithi116.stdout:(28/45): device-mapper-1.02.197-2.el9.x86_64.rp 1.4 MB/s | 139 kB 00:00 2024-02-18T11:49:02.152 INFO:teuthology.orchestra.run.smithi116.stdout:(29/45): python3-devel-3.9.18-3.el9.x86_64.rpm 1.6 MB/s | 204 kB 00:00 2024-02-18T11:49:02.178 INFO:teuthology.orchestra.run.smithi116.stdout:(30/45): device-mapper-event-1.02.197-2.el9.x86 448 kB/s | 33 kB 00:00 2024-02-18T11:49:02.203 INFO:teuthology.orchestra.run.smithi116.stdout:(31/45): device-mapper-event-libs-1.02.197-2.el 413 kB/s | 31 kB 00:00 2024-02-18T11:49:02.228 INFO:teuthology.orchestra.run.smithi116.stdout:(32/45): device-mapper-libs-1.02.197-2.el9.x86_ 3.5 MB/s | 178 kB 00:00 2024-02-18T11:49:02.253 INFO:teuthology.orchestra.run.smithi116.stdout:(33/45): e2fsprogs-1.46.5-5.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-02-18T11:49:02.279 INFO:teuthology.orchestra.run.smithi116.stdout:(34/45): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-02-18T11:49:02.304 INFO:teuthology.orchestra.run.smithi116.stdout:(35/45): libcom_err-1.46.5-5.el9.x86_64.rpm 511 kB/s | 25 kB 00:00 2024-02-18T11:49:02.329 INFO:teuthology.orchestra.run.smithi116.stdout:(36/45): libgcc-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 91 kB 00:00 2024-02-18T11:49:02.354 INFO:teuthology.orchestra.run.smithi116.stdout:(37/45): libgomp-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 267 kB 00:00 2024-02-18T11:49:02.380 INFO:teuthology.orchestra.run.smithi116.stdout:(38/45): libss-1.46.5-5.el9.x86_64.rpm 622 kB/s | 31 kB 00:00 2024-02-18T11:49:02.405 INFO:teuthology.orchestra.run.smithi116.stdout:(39/45): libstdc++-11.4.1-3.el9.x86_64.rpm 9.7 MB/s | 739 kB 00:00 2024-02-18T11:49:02.447 INFO:teuthology.orchestra.run.smithi116.stdout:(40/45): lvm2-2.03.23-2.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-02-18T11:49:02.489 INFO:teuthology.orchestra.run.smithi116.stdout:(41/45): make-4.3-8.el9.x86_64.rpm 6.4 MB/s | 529 kB 00:00 2024-02-18T11:49:02.522 INFO:teuthology.orchestra.run.smithi116.stdout:(42/45): lvm2-libs-2.03.23-2.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-02-18T11:49:02.548 INFO:teuthology.orchestra.run.smithi116.stdout:(43/45): python3-3.9.18-3.el9.x86_64.rpm 254 kB/s | 25 kB 00:00 2024-02-18T11:49:02.665 INFO:teuthology.orchestra.run.smithi116.stdout:(44/45): python3-libs-3.9.18-3.el9.x86_64.rpm 42 MB/s | 7.3 MB 00:00 2024-02-18T11:49:02.698 INFO:teuthology.orchestra.run.smithi116.stdout:(45/45): sqlite-libs-3.34.1-7.el9.x86_64.rpm 4.1 MB/s | 619 kB 00:00 2024-02-18T11:49:02.702 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-02-18T11:49:02.703 INFO:teuthology.orchestra.run.smithi116.stdout:Total 31 MB/s | 108 MB 00:03 2024-02-18T11:49:02.725 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-02-18T11:49:03.029 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-02-18T11:49:03.029 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-02-18T11:49:04.186 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-02-18T11:49:04.187 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-02-18T11:49:04.354 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:49:04.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:49:06.923 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-02-18T11:49:06.974 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/73 2024-02-18T11:49:07.020 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/73 2024-02-18T11:49:07.093 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/73 2024-02-18T11:49:07.154 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/73 2024-02-18T11:49:07.219 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/73 2024-02-18T11:49:07.272 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/73 2024-02-18T11:49:07.315 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/73 2024-02-18T11:49:07.358 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/73 2024-02-18T11:49:08.633 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 8/73 2024-02-18T11:49:08.727 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 9/73 2024-02-18T11:49:08.757 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 10/73 2024-02-18T11:49:08.808 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 11/73 2024-02-18T11:49:09.026 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : make-1:4.3-8.el9.x86_64 12/73 2024-02-18T11:49:09.978 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : git-core-2.43.0-1.el9.x86_64 13/73 2024-02-18T11:49:10.113 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : git-core-doc-2.43.0-1.el9.noarch 14/73 2024-02-18T11:49:10.149 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : perl-Git-2.43.0-1.el9.noarch 15/73 2024-02-18T11:49:10.185 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : git-2.43.0-1.el9.x86_64 16/73 2024-02-18T11:49:10.209 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-02-18T11:49:10.416 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-02-18T11:49:11.019 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 18/73 2024-02-18T11:49:11.116 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 19/73 2024-02-18T11:49:11.152 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 20/73 2024-02-18T11:49:11.519 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 21/73 2024-02-18T11:49:12.472 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 22/73 2024-02-18T11:49:12.644 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 23/73 2024-02-18T11:49:12.692 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 23/73 2024-02-18T11:49:12.728 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 24/73 2024-02-18T11:49:12.761 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rmt-2:1.6-6.el9.x86_64 25/73 2024-02-18T11:49:12.790 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 26/73 2024-02-18T11:49:12.831 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 27/73 2024-02-18T11:49:12.848 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 28/73 2024-02-18T11:49:12.874 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 29/73 2024-02-18T11:49:12.899 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 30/73 2024-02-18T11:49:12.924 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 31/73 2024-02-18T11:49:13.077 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 32/73 2024-02-18T11:49:13.116 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fio-3.35-1.el9.x86_64 33/73 2024-02-18T11:49:13.145 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 34/73 2024-02-18T11:49:22.427 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 35/73 2024-02-18T11:49:22.655 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : kernel-devel-5.14.0-421.el9.x86_64 36/73 2024-02-18T11:49:32.973 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: kernel-devel-5.14.0-421.el9.x86_64 36/73 2024-02-18T11:49:33.046 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 37/73 2024-02-18T11:49:33.070 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-02-18T11:49:33.168 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-02-18T11:49:33.378 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 39/73 2024-02-18T11:49:33.378 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-02-18T11:49:33.378 INFO:teuthology.orchestra.run.smithi116.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-02-18T11:49:33.378 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:49:33.665 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-02-18T11:49:33.710 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 41/73 2024-02-18T11:49:33.760 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 42/73 2024-02-18T11:49:33.797 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : indent-2.2.13-5.el9.x86_64 43/73 2024-02-18T11:49:33.839 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : udftools-2.2-5.el9.x86_64 44/73 2024-02-18T11:49:33.865 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 45/73 2024-02-18T11:49:33.892 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : git-2.39.3-1.el9.x86_64 46/73 2024-02-18T11:49:33.925 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 47/73 2024-02-18T11:49:33.961 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 48/73 2024-02-18T11:49:33.992 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 49/73 2024-02-18T11:49:34.017 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 50/73 2024-02-18T11:49:34.043 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 51/73 2024-02-18T11:49:34.050 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 52/73 2024-02-18T11:49:34.050 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 52/73 2024-02-18T11:49:34.102 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 52/73 2024-02-18T11:49:34.127 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 53/73 2024-02-18T11:49:34.132 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 54/73 2024-02-18T11:49:34.161 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 54/73 2024-02-18T11:49:34.180 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 55/73 2024-02-18T11:49:34.180 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 56/73 2024-02-18T11:49:34.224 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 56/73 2024-02-18T11:49:34.269 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 57/73 2024-02-18T11:49:34.296 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : git-core-doc-2.39.3-1.el9.noarch 58/73 2024-02-18T11:49:34.317 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : perl-Git-2.39.3-1.el9.noarch 59/73 2024-02-18T11:49:34.347 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 60/73 2024-02-18T11:49:34.372 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 61/73 2024-02-18T11:49:34.400 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 62/73 2024-02-18T11:49:34.417 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:49:34.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:49:34.428 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 63/73 2024-02-18T11:49:34.457 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 64/73 2024-02-18T11:49:34.543 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 65/73 2024-02-18T11:49:34.606 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 66/73 2024-02-18T11:49:34.627 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 67/73 2024-02-18T11:49:34.644 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 68/73 2024-02-18T11:49:34.645 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-02-18T11:49:34.730 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-02-18T11:49:34.754 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 70/73 2024-02-18T11:49:34.780 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 71/73 2024-02-18T11:49:34.808 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : make-1:4.3-7.el9.x86_64 72/73 2024-02-18T11:49:35.029 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : git-core-2.39.3-1.el9.x86_64 73/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: git-core-2.39.3-1.el9.x86_64 73/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-3.35-1.el9.x86_64 1/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/73 2024-02-18T11:49:37.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : kernel-devel-5.14.0-421.el9.x86_64 8/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 9/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 10/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 11/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : udftools-2.2-5.el9.x86_64 12/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 13/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/73 2024-02-18T11:49:37.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 18/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 19/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 20/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 21/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 22/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 23/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 24/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 25/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : git-2.43.0-1.el9.x86_64 26/73 2024-02-18T11:49:37.572 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : git-2.39.3-1.el9.x86_64 27/73 2024-02-18T11:49:37.573 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : git-core-2.43.0-1.el9.x86_64 28/73 2024-02-18T11:49:37.573 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : git-core-2.39.3-1.el9.x86_64 29/73 2024-02-18T11:49:37.577 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : git-core-doc-2.43.0-1.el9.noarch 30/73 2024-02-18T11:49:37.577 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : git-core-doc-2.39.3-1.el9.noarch 31/73 2024-02-18T11:49:37.577 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 32/73 2024-02-18T11:49:37.577 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 33/73 2024-02-18T11:49:37.578 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : perl-Git-2.43.0-1.el9.noarch 34/73 2024-02-18T11:49:37.578 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : perl-Git-2.39.3-1.el9.noarch 35/73 2024-02-18T11:49:37.578 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 36/73 2024-02-18T11:49:37.578 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 37/73 2024-02-18T11:49:37.578 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 38/73 2024-02-18T11:49:37.578 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 39/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 40/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 41/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 42/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 43/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 44/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 45/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 46/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 47/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 48/73 2024-02-18T11:49:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 49/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 50/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 51/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 52/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 53/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 54/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 55/73 2024-02-18T11:49:37.580 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 56/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 57/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libss-1.46.5-5.el9.x86_64 58/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libss-1.46.5-3.el9.x86_64 59/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 60/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 61/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 62/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 63/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 64/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 65/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : make-1:4.3-8.el9.x86_64 66/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : make-1:4.3-7.el9.x86_64 67/73 2024-02-18T11:49:37.581 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-3.9.18-3.el9.x86_64 68/73 2024-02-18T11:49:37.582 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-3.9.17-2.el9.x86_64 69/73 2024-02-18T11:49:37.582 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 70/73 2024-02-18T11:49:37.582 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 71/73 2024-02-18T11:49:37.582 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 72/73 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 73/73 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout:Upgraded: 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: cpp-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-02-18T11:49:38.707 INFO:teuthology.orchestra.run.smithi116.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: gcc-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: git-2.43.0-1.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: git-core-2.43.0-1.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: git-core-doc-2.43.0-1.el9.noarch 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: libgcc-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: libgomp-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: libss-1.46.5-5.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.708 INFO:teuthology.orchestra.run.smithi116.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: make-1:4.3-8.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: perl-Git-2.43.0-1.el9.noarch 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: python3-3.9.18-3.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-02-18T11:49:38.709 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-421.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: udftools-2.2-5.el9.x86_64 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:49:38.710 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-02-18T11:49:39.520 DEBUG:teuthology.orchestra.run.smithi116:> sudo useradd -m fsgqa 2024-02-18T11:49:39.549 INFO:teuthology.orchestra.run.smithi116.stderr:useradd: user 'fsgqa' already exists 2024-02-18T11:49:39.550 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-02-18T11:49:39.550 DEBUG:teuthology.orchestra.run.smithi116:> sudo groupadd fsgqa 2024-02-18T11:49:39.617 INFO:teuthology.orchestra.run.smithi116.stderr:groupadd: group 'fsgqa' already exists 2024-02-18T11:49:39.618 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-02-18T11:49:39.619 DEBUG:teuthology.orchestra.run.smithi116:> sudo useradd fsgqa2 2024-02-18T11:49:40.442 DEBUG:teuthology.orchestra.run.smithi116:> sudo useradd 123456-fsgqa 2024-02-18T11:49:40.881 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-02-18T11:49:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.991+0000 7f25d0c88640 1 Processor -- start 2024-02-18T11:49:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.991+0000 7f25d0c88640 1 -- start start 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25d0c88640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 0x7f25cc15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25d0c88640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f25cc15bdc0 0x7f25cc15c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25d0c88640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25cc06d6f0 con 0x7f25cc0b1740 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25d0c88640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc15c880 con 0x7f25cc15bdc0 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25d0c88640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc15cb60 con 0x7f25cc155e30 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25cad9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f25cc0b1740 0x7f25cc06db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59532/0 (socket says 172.21.15.116:59532) 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25cad9d640 1 -- 172.21.15.116:0/189108129 learned_addr learned my addr 172.21.15.116:0/189108129 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:49:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927701012 0 0) 0x7f25cc06d6f0 con 0x7f25cc0b1740 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b8003520 con 0x7f25cc0b1740 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c3fff640 1 --2- 172.21.15.116:0/189108129 >> v2:172.21.15.143:3300/0 conn(0x7f25cc15bdc0 0x7f25cc15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25ca59c640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 0x7f25cc15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3023894681 0 0) 0x7f25b8003520 con 0x7f25cc0b1740 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 msgr2=0x7f25cc15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 0x7f25cc15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 >> v2:172.21.15.143:3300/0 conn(0x7f25cc15bdc0 msgr2=0x7f25cc15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 --2- 172.21.15.116:0/189108129 >> v2:172.21.15.143:3300/0 conn(0x7f25cc15bdc0 0x7f25cc15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc06d380 con 0x7f25cc0b1740 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f25c40019f0 con 0x7f25cc0b1740 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c3fff640 1 --2- 172.21.15.116:0/189108129 >> v2:172.21.15.143:3300/0 conn(0x7f25cc15bdc0 0x7f25cc15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 >> v1:172.21.15.116:6789/0 conn(0x7f25cc0b1740 legacy=0x7f25cc06db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25b8003970 0x7f25b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 0x7f25b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25b8004d00 0x7f25b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc15c880 con 0x7f25b8003970 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc15cb60 con 0x7f25b8004d00 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b800ba30 con 0x7f25cc155e30 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.992+0000 7f25ca59c640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25b8003970 0x7f25b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.993+0000 7f25ca59c640 1 -- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 msgr2=0x7f25b80047c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.993+0000 7f25ca59c640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25cc155e30 0x7f25b80047c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.993+0000 7f25ca59c640 1 -- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25b8004d00 msgr2=0x7f25b8005180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.993+0000 7f25ca59c640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25b8004d00 0x7f25b8005180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.993+0000 7f25ca59c640 1 -- 172.21.15.116:0/189108129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8003520 con 0x7f25b8003970 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.993+0000 7f25ca59c640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25b8003970 0x7f25b8003d90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00ba60 tx=0x7f25b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69430ed63dfb8013 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc032640 con 0x7f25b8003970 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc0327a0 con 0x7f25b8003970 2024-02-18T11:49:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25c9d9b640 1 -- 172.21.15.116:0/189108129 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc032b30 con 0x7f25b8003970 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25d0c88640 1 -- 172.21.15.116:0/189108129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25b8003970 msgr2=0x7f25b8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25d0c88640 1 --2- 172.21.15.116:0/189108129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25b8003970 0x7f25b8003d90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00ba60 tx=0x7f25b80057f0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25cad9d640 1 -- 172.21.15.116:0/189108129 reap_dead start 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25d0c88640 1 -- 172.21.15.116:0/189108129 shutdown_connections 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25d0c88640 1 -- 172.21.15.116:0/189108129 >> 172.21.15.116:0/189108129 conn(0x7f25cc066f20 msgr2=0x7f25cc06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25d0c88640 1 -- 172.21.15.116:0/189108129 shutdown_connections 2024-02-18T11:49:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.994+0000 7f25d0c88640 1 -- 172.21.15.116:0/189108129 wait complete. 2024-02-18T11:49:40.992 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 Processor -- start 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 -- start start 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25b8003970 0x7f25cc157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0a98f0 0x7f25cc0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 0x7f25cc15fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1600e0 con 0x7f25cc15f720 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1f25b0 con 0x7f25cc0a98f0 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25d0c88640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc0ade00 con 0x7f25b8003970 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25cad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 0x7f25cc15fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25cad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 0x7f25cc15fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47496/0 (socket says 172.21.15.116:47496) 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25cad9d640 1 -- 172.21.15.116:0/3856189018 learned_addr learned my addr 172.21.15.116:0/3856189018 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.996+0000 7f25ca59c640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25b8003970 0x7f25cc157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25cad9d640 1 -- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25b8003970 msgr2=0x7f25cc157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25cad9d640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25b8003970 0x7f25cc157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25cad9d640 1 -- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0a98f0 msgr2=0x7f25cc0ae050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25cad9d640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0a98f0 0x7f25cc0ae050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25cad9d640 1 -- 172.21.15.116:0/3856189018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc1f1a30 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25cad9d640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 0x7f25cc15fba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06f2c0 tx=0x7f25c400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400a610 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400a770 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400ab20 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc1f1d10 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.997+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc1f22d0 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.998+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc074520 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.998+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f25c400ac80 con 0x7f25cc15f720 2024-02-18T11:49:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.999+0000 7f25c2ffd640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ac06bc70 0x7f25ac06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:40.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.999+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f25c408a0f0 con 0x7f25cc15f720 2024-02-18T11:49:40.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.999+0000 7f25ca59c640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ac06bc70 0x7f25ac06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:40.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:40.999+0000 7f25ca59c640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ac06bc70 0x7f25ac06e130 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00ba30 tx=0x7f25bc03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:49:40.995 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.003+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25c4054720 con 0x7f25cc15f720 2024-02-18T11:49:41.232 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.240+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f25cc1559a0 con 0x7f25cc15f720 2024-02-18T11:49:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.240+0000 7f25c2ffd640 1 -- 172.21.15.116:0/3856189018 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1289 (secure 0 0 0) 0x7f25c4057de0 con 0x7f25cc15f720 2024-02-18T11:49:41.233 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:49:41.233 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":1,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","modified":"2024-02-18T11:46:35.744646Z","created":"2024-02-18T11:46:35.744646Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3301","nonce":0},{"type":"v1","addr":"172.21.15.143:6790","nonce":0}]},"addr":"172.21.15.143:6790/0","public_addr":"172.21.15.143:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-18T11:49:41.233 INFO:teuthology.orchestra.run.smithi116.stderr:dumped monmap epoch 1 2024-02-18T11:49:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ac06bc70 msgr2=0x7f25ac06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ac06bc70 0x7f25ac06e130 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00ba30 tx=0x7f25bc03d040 comp rx=0 tx=0).stop 2024-02-18T11:49:41.235 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 msgr2=0x7f25cc15fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 0x7f25cc15fba0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f25cc06f2c0 tx=0x7f25c400cf00 comp rx=0 tx=0).stop 2024-02-18T11:49:41.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 shutdown_connections 2024-02-18T11:49:41.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f25ac06bc70 0x7f25ac06e130 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25cc15f720 0x7f25cc15fba0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0a98f0 0x7f25cc0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.236 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 --2- 172.21.15.116:0/3856189018 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f25b8003970 0x7f25cc157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 >> 172.21.15.116:0/3856189018 conn(0x7f25cc066f20 msgr2=0x7f25cc0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:49:41.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 shutdown_connections 2024-02-18T11:49:41.237 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.242+0000 7f25d0c88640 1 -- 172.21.15.116:0/3856189018 wait complete. 2024-02-18T11:49:41.247 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-02-18T11:49:41.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.356+0000 7f0451ec0640 1 Processor -- start 2024-02-18T11:49:41.348 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.356+0000 7f0451ec0640 1 -- start start 2024-02-18T11:49:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f0451ec0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f044c0cdd20 0x7f044c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f0451ec0640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f044c071f90 0x7f044c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f0451ec0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044c15a480 con 0x7f044c150ed0 2024-02-18T11:49:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f0451ec0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c072420 con 0x7f044c0cdd20 2024-02-18T11:49:41.350 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f0451ec0640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c15a760 con 0x7f044c071f90 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f044c0cdd20 0x7f044c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044b7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f044c150ed0 0x7f044c0b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59544/0 (socket says 172.21.15.116:59544) 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f044c0cdd20 0x7f044c071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47510/0 (socket says 172.21.15.116:47510) 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044b7fe640 1 -- 172.21.15.116:0/2263604917 learned_addr learned my addr 172.21.15.116:0/2263604917 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044bfff640 1 --2- 172.21.15.116:0/2263604917 >> v2:172.21.15.143:3300/0 conn(0x7f044c071f90 0x7f044c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044affd640 1 -- 172.21.15.116:0/2263604917 >> v2:172.21.15.143:3300/0 conn(0x7f044c071f90 msgr2=0x7f044c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044affd640 1 --2- 172.21.15.116:0/2263604917 >> v2:172.21.15.143:3300/0 conn(0x7f044c071f90 0x7f044c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044affd640 1 -- 172.21.15.116:0/2263604917 >> v1:172.21.15.116:6789/0 conn(0x7f044c150ed0 legacy=0x7f044c0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.357+0000 7f044affd640 1 -- 172.21.15.116:0/2263604917 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c15aa40 con 0x7f044c0cdd20 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.358+0000 7f044affd640 1 --2- 172.21.15.116:0/2263604917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c071a50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f04400097a0 tx=0x7f044002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e382e1843d3f77a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.358+0000 7f044a7fc640 1 -- 172.21.15.116:0/2263604917 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044002e070 con 0x7f044c0cdd20 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.358+0000 7f044a7fc640 1 -- 172.21.15.116:0/2263604917 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04400315a0 con 0x7f044c0cdd20 2024-02-18T11:49:41.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.358+0000 7f044a7fc640 1 -- 172.21.15.116:0/2263604917 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440031930 con 0x7f044c0cdd20 2024-02-18T11:49:41.352 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 -- 172.21.15.116:0/2263604917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 msgr2=0x7f044c071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2263604917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c071a50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f04400097a0 tx=0x7f044002b840 comp rx=0 tx=0).stop 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 -- 172.21.15.116:0/2263604917 shutdown_connections 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2263604917 >> v2:172.21.15.143:3300/0 conn(0x7f044c071f90 0x7f044c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2263604917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c071a50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 -- 172.21.15.116:0/2263604917 >> 172.21.15.116:0/2263604917 conn(0x7f044c067000 msgr2=0x7f044c06ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 -- 172.21.15.116:0/2263604917 shutdown_connections 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.359+0000 7f0451ec0640 1 -- 172.21.15.116:0/2263604917 wait complete. 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.360+0000 7f0451ec0640 1 Processor -- start 2024-02-18T11:49:41.353 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.360+0000 7f0451ec0640 1 -- start start 2024-02-18T11:49:41.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f0451ec0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f044c071f90 0x7f044c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f0451ec0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f0451ec0640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f044c150ed0 0x7f044c0ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f0451ec0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c15def0 con 0x7f044c0cdd20 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f0451ec0640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c15d2e0 con 0x7f044c071f90 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f0451ec0640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c15d5c0 con 0x7f044c150ed0 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c15ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47522/0 (socket says 172.21.15.116:47522) 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 -- 172.21.15.116:0/2262528509 learned_addr learned my addr 172.21.15.116:0/2262528509 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044bfff640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f044c150ed0 0x7f044c0ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 -- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f044c150ed0 msgr2=0x7f044c0ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f044c150ed0 0x7f044c0ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 -- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f044c071f90 msgr2=0x7f044c15e380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f044c071f90 0x7f044c15e380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.361+0000 7f044affd640 1 -- 172.21.15.116:0/2262528509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440009400 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.362+0000 7f044affd640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c15ca90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0440034040 tx=0x7f04400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.362+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440032840 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.362+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04400329c0 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.362+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c15d8a0 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.362+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440032d70 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.362+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c15e8c0 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.363+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044c074520 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.363+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f044004e430 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.363+0000 7f0433fff640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f043806bc70 0x7f043806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.364+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f04400ac9c0 con 0x7f044c0cdd20 2024-02-18T11:49:41.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.364+0000 7f044b7fe640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f043806bc70 0x7f043806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T11:49:41.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.364+0000 7f044b7fe640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f043806bc70 0x7f043806e130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f043c000f60 tx=0x7f043c007a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T11:49:41.360 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.368+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0440076ff0 con 0x7f044c0cdd20 2024-02-18T11:49:41.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.606+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7f044c14ba40 con 0x7f044c0cdd20 2024-02-18T11:49:41.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.606+0000 7f0433fff640 1 -- 172.21.15.116:0/2262528509 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v11) v1 ==== 92+0+63 (secure 0 0 0) 0x7f044007a6b0 con 0x7f044c0cdd20 2024-02-18T11:49:41.598 INFO:teuthology.orchestra.run.smithi116.stdout:[client.admin] 2024-02-18T11:49:41.598 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== 2024-02-18T11:49:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f043806bc70 msgr2=0x7f043806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f043806bc70 0x7f043806e130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f043c000f60 tx=0x7f043c007a00 comp rx=0 tx=0).stop 2024-02-18T11:49:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 msgr2=0x7f044c15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T11:49:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c15ca90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0440034040 tx=0x7f04400046c0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 shutdown_connections 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f043806bc70 0x7f043806e130 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f044c150ed0 0x7f044c0ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f044c0cdd20 0x7f044c15ca90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 --2- 172.21.15.116:0/2262528509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f044c071f90 0x7f044c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 >> 172.21.15.116:0/2262528509 conn(0x7f044c067000 msgr2=0x7f044c1588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 shutdown_connections 2024-02-18T11:49:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T11:49:41.608+0000 7f0451ec0640 1 -- 172.21.15.116:0/2262528509 wait complete. 2024-02-18T11:49:41.612 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:49:41.613 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/tmp/tmp.hC7pXm5hUExfstests-dev/local.config 2024-02-18T11:49:41.639 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.116:6789:/test export TEST_DIR=/tmp/tmp.jguMaGqYX4test export SCRATCH_DEV=172.21.15.116:6789:/scratch export SCRATCH_MNT=/tmp/tmp.gu5YvuvMAWscratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg==" 2024-02-18T11:49:41.640 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T11:49:41.640 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/tmp/tmp.hC7pXm5hUExfstests-dev/ceph.exclude 2024-02-18T11:49:41.706 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo make) 2024-02-18T11:49:41.778 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize -cfi 2024-02-18T11:49:42.202 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: putting auxiliary files in '.'. 2024-02-18T11:49:42.202 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file './config.guess' 2024-02-18T11:49:42.205 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file './config.sub' 2024-02-18T11:49:42.209 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file './install-sh' 2024-02-18T11:49:42.212 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file './ltmain.sh' 2024-02-18T11:49:42.234 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-02-18T11:49:42.235 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-02-18T11:49:42.239 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-02-18T11:49:42.244 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-02-18T11:49:42.247 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-02-18T11:49:42.252 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-02-18T11:49:42.258 INFO:teuthology.orchestra.run.smithi116.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-02-18T11:49:42.259 INFO:teuthology.orchestra.run.smithi116.stdout:cp include/install-sh . 2024-02-18T11:49:42.260 INFO:teuthology.orchestra.run.smithi116.stdout:aclocal -I m4 2024-02-18T11:49:43.285 INFO:teuthology.orchestra.run.smithi116.stdout:autoheader 2024-02-18T11:49:43.586 INFO:teuthology.orchestra.run.smithi116.stdout:autoconf 2024-02-18T11:49:43.751 INFO:teuthology.orchestra.run.smithi116.stdout:./configure \ 2024-02-18T11:49:43.751 INFO:teuthology.orchestra.run.smithi116.stdout: --libexecdir=/usr/lib \ 2024-02-18T11:49:43.751 INFO:teuthology.orchestra.run.smithi116.stdout: --exec_prefix=/var/lib 2024-02-18T11:49:43.965 INFO:teuthology.orchestra.run.smithi116.stdout:checking build system type... x86_64-pc-linux-gnu 2024-02-18T11:49:43.965 INFO:teuthology.orchestra.run.smithi116.stdout:checking host system type... x86_64-pc-linux-gnu 2024-02-18T11:49:43.966 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to print strings... printf 2024-02-18T11:49:43.966 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gcc... gcc 2024-02-18T11:49:44.027 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether the C compiler works... yes 2024-02-18T11:49:44.027 INFO:teuthology.orchestra.run.smithi116.stdout:checking for C compiler default output file name... a.out 2024-02-18T11:49:44.054 INFO:teuthology.orchestra.run.smithi116.stdout:checking for suffix of executables... 2024-02-18T11:49:44.114 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether we are cross compiling... no 2024-02-18T11:49:44.131 INFO:teuthology.orchestra.run.smithi116.stdout:checking for suffix of object files... o 2024-02-18T11:49:44.146 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether we are using the GNU C compiler... yes 2024-02-18T11:49:44.161 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether gcc accepts -g... yes 2024-02-18T11:49:44.187 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gcc option to accept ISO C89... none needed 2024-02-18T11:49:44.189 INFO:teuthology.orchestra.run.smithi116.stdout:checking for a sed that does not truncate output... /bin/sed 2024-02-18T11:49:44.191 INFO:teuthology.orchestra.run.smithi116.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-02-18T11:49:44.192 INFO:teuthology.orchestra.run.smithi116.stdout:checking for egrep... /bin/grep -E 2024-02-18T11:49:44.193 INFO:teuthology.orchestra.run.smithi116.stdout:checking for fgrep... /bin/grep -F 2024-02-18T11:49:44.197 INFO:teuthology.orchestra.run.smithi116.stdout:checking for ld used by gcc... /bin/ld 2024-02-18T11:49:44.198 INFO:teuthology.orchestra.run.smithi116.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-02-18T11:49:44.205 INFO:teuthology.orchestra.run.smithi116.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-02-18T11:49:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-02-18T11:49:45.115 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether ln -s works... yes 2024-02-18T11:49:45.128 INFO:teuthology.orchestra.run.smithi116.stdout:checking the maximum length of command line arguments... 1572864 2024-02-18T11:49:45.133 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-02-18T11:49:45.133 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-02-18T11:49:45.133 INFO:teuthology.orchestra.run.smithi116.stdout:checking for /bin/ld option to reload object files... -r 2024-02-18T11:49:45.133 INFO:teuthology.orchestra.run.smithi116.stdout:checking for objdump... objdump 2024-02-18T11:49:45.133 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to recognize dependent libraries... pass_all 2024-02-18T11:49:45.133 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dlltool... no 2024-02-18T11:49:45.134 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-02-18T11:49:45.134 INFO:teuthology.orchestra.run.smithi116.stdout:checking for ar... ar 2024-02-18T11:49:45.187 INFO:teuthology.orchestra.run.smithi116.stdout:checking for archiver @FILE support... @ 2024-02-18T11:49:45.187 INFO:teuthology.orchestra.run.smithi116.stdout:checking for strip... strip 2024-02-18T11:49:45.187 INFO:teuthology.orchestra.run.smithi116.stdout:checking for ranlib... ranlib 2024-02-18T11:49:45.188 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gawk... gawk 2024-02-18T11:49:45.253 INFO:teuthology.orchestra.run.smithi116.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-02-18T11:49:45.254 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sysroot... no 2024-02-18T11:49:45.256 INFO:teuthology.orchestra.run.smithi116.stdout:checking for a working dd... /bin/dd 2024-02-18T11:49:45.259 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-02-18T11:49:45.272 INFO:teuthology.orchestra.run.smithi116.stdout:checking for mt... no 2024-02-18T11:49:45.273 INFO:teuthology.orchestra.run.smithi116.stdout:checking if : is a manifest tool... no 2024-02-18T11:49:45.324 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to run the C preprocessor... gcc -E 2024-02-18T11:49:45.488 INFO:teuthology.orchestra.run.smithi116.stdout:checking for ANSI C header files... yes 2024-02-18T11:49:45.508 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/types.h... yes 2024-02-18T11:49:45.540 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/stat.h... yes 2024-02-18T11:49:45.563 INFO:teuthology.orchestra.run.smithi116.stdout:checking for stdlib.h... yes 2024-02-18T11:49:45.587 INFO:teuthology.orchestra.run.smithi116.stdout:checking for string.h... yes 2024-02-18T11:49:45.616 INFO:teuthology.orchestra.run.smithi116.stdout:checking for memory.h... yes 2024-02-18T11:49:45.641 INFO:teuthology.orchestra.run.smithi116.stdout:checking for strings.h... yes 2024-02-18T11:49:45.675 INFO:teuthology.orchestra.run.smithi116.stdout:checking for inttypes.h... yes 2024-02-18T11:49:45.699 INFO:teuthology.orchestra.run.smithi116.stdout:checking for stdint.h... yes 2024-02-18T11:49:45.730 INFO:teuthology.orchestra.run.smithi116.stdout:checking for unistd.h... yes 2024-02-18T11:49:45.759 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dlfcn.h... yes 2024-02-18T11:49:45.762 INFO:teuthology.orchestra.run.smithi116.stdout:checking for objdir... .libs 2024-02-18T11:49:45.824 INFO:teuthology.orchestra.run.smithi116.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-02-18T11:49:45.824 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-02-18T11:49:45.838 INFO:teuthology.orchestra.run.smithi116.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-02-18T11:49:45.856 INFO:teuthology.orchestra.run.smithi116.stdout:checking if gcc static flag -static works... no 2024-02-18T11:49:45.877 INFO:teuthology.orchestra.run.smithi116.stdout:checking if gcc supports -c -o file.o... yes 2024-02-18T11:49:45.877 INFO:teuthology.orchestra.run.smithi116.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-02-18T11:49:45.884 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-02-18T11:49:45.914 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether -lc should be explicitly linked in... no 2024-02-18T11:49:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-02-18T11:49:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:checking how to hardcode library paths into programs... immediate 2024-02-18T11:49:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether stripping libraries is possible... yes 2024-02-18T11:49:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:checking if libtool supports shared libraries... yes 2024-02-18T11:49:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether to build shared libraries... yes 2024-02-18T11:49:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether to build static libraries... yes 2024-02-18T11:49:45.983 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gcc... (cached) gcc 2024-02-18T11:49:45.998 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-02-18T11:49:45.998 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether gcc accepts -g... (cached) yes 2024-02-18T11:49:45.998 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-02-18T11:49:45.998 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gmake... /usr/bin/gmake 2024-02-18T11:49:45.998 INFO:teuthology.orchestra.run.smithi116.stderr:./configure: line 12083: test: $(SHELL): binary operator expected 2024-02-18T11:49:45.999 INFO:teuthology.orchestra.run.smithi116.stdout:checking for tar... /bin/tar 2024-02-18T11:49:45.999 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gzip... /bin/gzip 2024-02-18T11:49:46.007 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether gcc -MM is supported... yes 2024-02-18T11:49:46.007 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sort... /bin/sort 2024-02-18T11:49:46.007 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether ln -s works... yes 2024-02-18T11:49:46.007 INFO:teuthology.orchestra.run.smithi116.stdout:checking for rpm... /bin/rpm 2024-02-18T11:49:46.010 INFO:teuthology.orchestra.run.smithi116.stdout:checking for rpmbuild... no 2024-02-18T11:49:46.035 INFO:teuthology.orchestra.run.smithi116.stdout:checking assert.h usability... yes 2024-02-18T11:49:46.044 INFO:teuthology.orchestra.run.smithi116.stdout:checking assert.h presence... yes 2024-02-18T11:49:46.044 INFO:teuthology.orchestra.run.smithi116.stdout:checking for assert.h... yes 2024-02-18T11:49:46.069 INFO:teuthology.orchestra.run.smithi116.stdout:checking bstring.h usability... no 2024-02-18T11:49:46.079 INFO:teuthology.orchestra.run.smithi116.stdout:checking bstring.h presence... no 2024-02-18T11:49:46.079 INFO:teuthology.orchestra.run.smithi116.stdout:checking for bstring.h... no 2024-02-18T11:49:46.104 INFO:teuthology.orchestra.run.smithi116.stdout:checking libgen.h usability... yes 2024-02-18T11:49:46.112 INFO:teuthology.orchestra.run.smithi116.stdout:checking libgen.h presence... yes 2024-02-18T11:49:46.112 INFO:teuthology.orchestra.run.smithi116.stdout:checking for libgen.h... yes 2024-02-18T11:49:46.149 INFO:teuthology.orchestra.run.smithi116.stdout:checking dirent.h usability... yes 2024-02-18T11:49:46.158 INFO:teuthology.orchestra.run.smithi116.stdout:checking dirent.h presence... yes 2024-02-18T11:49:46.158 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dirent.h... yes 2024-02-18T11:49:46.214 INFO:teuthology.orchestra.run.smithi116.stdout:checking errno.h usability... yes 2024-02-18T11:49:46.223 INFO:teuthology.orchestra.run.smithi116.stdout:checking errno.h presence... yes 2024-02-18T11:49:46.223 INFO:teuthology.orchestra.run.smithi116.stdout:checking for errno.h... yes 2024-02-18T11:49:46.255 INFO:teuthology.orchestra.run.smithi116.stdout:checking malloc.h usability... yes 2024-02-18T11:49:46.268 INFO:teuthology.orchestra.run.smithi116.stdout:checking malloc.h presence... yes 2024-02-18T11:49:46.268 INFO:teuthology.orchestra.run.smithi116.stdout:checking for malloc.h... yes 2024-02-18T11:49:46.291 INFO:teuthology.orchestra.run.smithi116.stdout:checking uuid.h usability... no 2024-02-18T11:49:46.301 INFO:teuthology.orchestra.run.smithi116.stdout:checking uuid.h presence... no 2024-02-18T11:49:46.301 INFO:teuthology.orchestra.run.smithi116.stdout:checking for uuid.h... no 2024-02-18T11:49:46.344 INFO:teuthology.orchestra.run.smithi116.stdout:checking uuid/uuid.h usability... yes 2024-02-18T11:49:46.354 INFO:teuthology.orchestra.run.smithi116.stdout:checking uuid/uuid.h presence... yes 2024-02-18T11:49:46.355 INFO:teuthology.orchestra.run.smithi116.stdout:checking for uuid/uuid.h... yes 2024-02-18T11:49:46.380 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/uuid.h usability... no 2024-02-18T11:49:46.391 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/uuid.h presence... no 2024-02-18T11:49:46.391 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/uuid.h... no 2024-02-18T11:49:46.423 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/file.h usability... yes 2024-02-18T11:49:46.432 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/file.h presence... yes 2024-02-18T11:49:46.432 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/file.h... yes 2024-02-18T11:49:46.467 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/fcntl.h usability... yes 2024-02-18T11:49:46.476 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/fcntl.h presence... yes 2024-02-18T11:49:46.476 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/fcntl.h... yes 2024-02-18T11:49:46.502 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/syssgi.h usability... no 2024-02-18T11:49:46.513 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/syssgi.h presence... no 2024-02-18T11:49:46.513 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/syssgi.h... no 2024-02-18T11:49:46.556 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/param.h usability... yes 2024-02-18T11:49:46.568 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/param.h presence... yes 2024-02-18T11:49:46.568 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/param.h... yes 2024-02-18T11:49:46.573 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/stat.h... (cached) yes 2024-02-18T11:49:46.615 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/statvfs.h usability... yes 2024-02-18T11:49:46.624 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/statvfs.h presence... yes 2024-02-18T11:49:46.624 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/statvfs.h... yes 2024-02-18T11:49:46.651 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/time.h usability... yes 2024-02-18T11:49:46.659 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/time.h presence... yes 2024-02-18T11:49:46.660 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/time.h... yes 2024-02-18T11:49:46.691 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/ioctl.h usability... yes 2024-02-18T11:49:46.700 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/ioctl.h presence... yes 2024-02-18T11:49:46.700 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/ioctl.h... yes 2024-02-18T11:49:46.729 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/wait.h usability... yes 2024-02-18T11:49:46.739 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/wait.h presence... yes 2024-02-18T11:49:46.739 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/wait.h... yes 2024-02-18T11:49:46.743 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/types.h... (cached) yes 2024-02-18T11:49:46.748 INFO:teuthology.orchestra.run.smithi116.stdout:checking for strings.h... (cached) yes 2024-02-18T11:49:46.776 INFO:teuthology.orchestra.run.smithi116.stdout:checking err.h usability... yes 2024-02-18T11:49:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:checking err.h presence... yes 2024-02-18T11:49:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:checking for err.h... yes 2024-02-18T11:49:46.810 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/falloc.h usability... yes 2024-02-18T11:49:46.818 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/falloc.h presence... yes 2024-02-18T11:49:46.818 INFO:teuthology.orchestra.run.smithi116.stdout:checking for linux/falloc.h... yes 2024-02-18T11:49:46.843 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-02-18T11:49:46.854 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-02-18T11:49:46.854 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/fs/xfs_fsops.h... no 2024-02-18T11:49:46.877 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/fs/xfs_itable.h usability... no 2024-02-18T11:49:46.888 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/fs/xfs_itable.h presence... no 2024-02-18T11:49:46.888 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/fs/xfs_itable.h... no 2024-02-18T11:49:46.911 INFO:teuthology.orchestra.run.smithi116.stdout:checking xfs/platform_defs.h usability... no 2024-02-18T11:49:46.921 INFO:teuthology.orchestra.run.smithi116.stdout:checking xfs/platform_defs.h presence... no 2024-02-18T11:49:46.922 INFO:teuthology.orchestra.run.smithi116.stdout:checking for xfs/platform_defs.h... no 2024-02-18T11:49:46.946 INFO:teuthology.orchestra.run.smithi116.stdout:checking btrfs/ioctl.h usability... no 2024-02-18T11:49:46.956 INFO:teuthology.orchestra.run.smithi116.stdout:checking btrfs/ioctl.h presence... no 2024-02-18T11:49:46.957 INFO:teuthology.orchestra.run.smithi116.stdout:checking for btrfs/ioctl.h... no 2024-02-18T11:49:46.980 INFO:teuthology.orchestra.run.smithi116.stdout:checking cifs/ioctl.h usability... no 2024-02-18T11:49:46.990 INFO:teuthology.orchestra.run.smithi116.stdout:checking cifs/ioctl.h presence... no 2024-02-18T11:49:46.990 INFO:teuthology.orchestra.run.smithi116.stdout:checking for cifs/ioctl.h... no 2024-02-18T11:49:47.032 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/mman.h usability... yes 2024-02-18T11:49:47.041 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/mman.h presence... yes 2024-02-18T11:49:47.041 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/mman.h... yes 2024-02-18T11:49:47.055 INFO:teuthology.orchestra.run.smithi116.stdout:checking for xfs/xfs_log_format.h... no 2024-02-18T11:49:47.089 INFO:teuthology.orchestra.run.smithi116.stdout:checking for uuid_compare... no 2024-02-18T11:49:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:checking for uuid_compare in -luuid... yes 2024-02-18T11:49:47.188 INFO:teuthology.orchestra.run.smithi116.stdout:checking for xfs/xfs.h... yes 2024-02-18T11:49:47.201 INFO:teuthology.orchestra.run.smithi116.stdout:checking for xfs/libxfs.h... no 2024-02-18T11:49:47.215 INFO:teuthology.orchestra.run.smithi116.stdout:checking whether xlog_assign_lsn is declared... no 2024-02-18T11:49:47.244 INFO:teuthology.orchestra.run.smithi116.stdout:checking for xfs/xqm.h... yes 2024-02-18T11:49:47.292 INFO:teuthology.orchestra.run.smithi116.stdout:checking xfsctl from xfs/xfs.h... ok 2024-02-18T11:49:47.315 INFO:teuthology.orchestra.run.smithi116.stdout:checking xfs/handle.h usability... yes 2024-02-18T11:49:47.322 INFO:teuthology.orchestra.run.smithi116.stdout:checking xfs/handle.h presence... yes 2024-02-18T11:49:47.322 INFO:teuthology.orchestra.run.smithi116.stdout:checking for xfs/handle.h... yes 2024-02-18T11:49:47.353 INFO:teuthology.orchestra.run.smithi116.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-02-18T11:49:47.376 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/acl.h usability... yes 2024-02-18T11:49:47.385 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/acl.h presence... yes 2024-02-18T11:49:47.385 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/acl.h... yes 2024-02-18T11:49:47.408 INFO:teuthology.orchestra.run.smithi116.stdout:checking acl/libacl.h usability... yes 2024-02-18T11:49:47.418 INFO:teuthology.orchestra.run.smithi116.stdout:checking acl/libacl.h presence... yes 2024-02-18T11:49:47.418 INFO:teuthology.orchestra.run.smithi116.stdout:checking for acl/libacl.h... yes 2024-02-18T11:49:47.449 INFO:teuthology.orchestra.run.smithi116.stdout:checking for acl_init in -lacl... yes 2024-02-18T11:49:47.472 INFO:teuthology.orchestra.run.smithi116.stdout:checking gdbm-ndbm.h usability... no 2024-02-18T11:49:47.482 INFO:teuthology.orchestra.run.smithi116.stdout:checking gdbm-ndbm.h presence... no 2024-02-18T11:49:47.483 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gdbm-ndbm.h... no 2024-02-18T11:49:47.507 INFO:teuthology.orchestra.run.smithi116.stdout:checking gdbm/ndbm.h usability... yes 2024-02-18T11:49:47.515 INFO:teuthology.orchestra.run.smithi116.stdout:checking gdbm/ndbm.h presence... yes 2024-02-18T11:49:47.515 INFO:teuthology.orchestra.run.smithi116.stdout:checking for gdbm/ndbm.h... yes 2024-02-18T11:49:47.549 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_open in -lgdbm... no 2024-02-18T11:49:47.583 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_fetch in -lgdbm... no 2024-02-18T11:49:47.616 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_store in -lgdbm... no 2024-02-18T11:49:47.650 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_close in -lgdbm... no 2024-02-18T11:49:47.684 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-02-18T11:49:47.715 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-02-18T11:49:47.747 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-02-18T11:49:47.778 INFO:teuthology.orchestra.run.smithi116.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-02-18T11:49:47.802 INFO:teuthology.orchestra.run.smithi116.stdout:checking libaio.h usability... yes 2024-02-18T11:49:47.812 INFO:teuthology.orchestra.run.smithi116.stdout:checking libaio.h presence... yes 2024-02-18T11:49:47.812 INFO:teuthology.orchestra.run.smithi116.stdout:checking for libaio.h... yes 2024-02-18T11:49:47.836 INFO:teuthology.orchestra.run.smithi116.stdout:checking liburing.h usability... no 2024-02-18T11:49:47.846 INFO:teuthology.orchestra.run.smithi116.stdout:checking liburing.h presence... no 2024-02-18T11:49:47.846 INFO:teuthology.orchestra.run.smithi116.stdout:checking for liburing.h... no 2024-02-18T11:49:47.871 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/capability.h usability... yes 2024-02-18T11:49:47.880 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/capability.h presence... yes 2024-02-18T11:49:47.881 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/capability.h... yes 2024-02-18T11:49:47.904 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/fiemap.h usability... yes 2024-02-18T11:49:47.912 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/fiemap.h presence... yes 2024-02-18T11:49:47.912 INFO:teuthology.orchestra.run.smithi116.stdout:checking for linux/fiemap.h... yes 2024-02-18T11:49:47.945 INFO:teuthology.orchestra.run.smithi116.stdout:checking for fallocate... yes 2024-02-18T11:49:47.979 INFO:teuthology.orchestra.run.smithi116.stdout:checking for open_by_handle_at... yes 2024-02-18T11:49:48.003 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/prctl.h usability... yes 2024-02-18T11:49:48.011 INFO:teuthology.orchestra.run.smithi116.stdout:checking sys/prctl.h presence... yes 2024-02-18T11:49:48.011 INFO:teuthology.orchestra.run.smithi116.stdout:checking for sys/prctl.h... yes 2024-02-18T11:49:48.035 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/fs.h usability... yes 2024-02-18T11:49:48.044 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/fs.h presence... yes 2024-02-18T11:49:48.044 INFO:teuthology.orchestra.run.smithi116.stdout:checking for linux/fs.h... yes 2024-02-18T11:49:48.065 INFO:teuthology.orchestra.run.smithi116.stdout:checking btrfsutil.h usability... no 2024-02-18T11:49:48.076 INFO:teuthology.orchestra.run.smithi116.stdout:checking btrfsutil.h presence... no 2024-02-18T11:49:48.076 INFO:teuthology.orchestra.run.smithi116.stdout:checking for btrfsutil.h... no 2024-02-18T11:49:48.116 INFO:teuthology.orchestra.run.smithi116.stdout:checking for copy_file_range... yes 2024-02-18T11:49:48.151 INFO:teuthology.orchestra.run.smithi116.stdout:checking for SEEK_DATA... yes 2024-02-18T11:49:48.200 INFO:teuthology.orchestra.run.smithi116.stdout:checking for BMV_OF_SHARED... yes 2024-02-18T11:49:48.236 INFO:teuthology.orchestra.run.smithi116.stdout:checking for nftw... yes 2024-02-18T11:49:48.278 INFO:teuthology.orchestra.run.smithi116.stdout:checking for RLIMIT_NOFILE... yes 2024-02-18T11:49:48.310 INFO:teuthology.orchestra.run.smithi116.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-02-18T11:49:48.343 INFO:teuthology.orchestra.run.smithi116.stdout:checking for FICLONE... yes 2024-02-18T11:49:48.376 INFO:teuthology.orchestra.run.smithi116.stdout:checking for renameat2... yes 2024-02-18T11:49:48.409 INFO:teuthology.orchestra.run.smithi116.stdout:checking for reallocarray... yes 2024-02-18T11:49:48.439 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct mount_attr... yes 2024-02-18T11:49:48.473 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_qgroup_limit... yes 2024-02-18T11:49:48.504 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-02-18T11:49:48.536 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-02-18T11:49:48.568 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-02-18T11:49:48.600 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-02-18T11:49:48.631 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-02-18T11:49:48.662 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-02-18T11:49:48.688 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/btrfs.h usability... yes 2024-02-18T11:49:48.696 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/btrfs.h presence... yes 2024-02-18T11:49:48.696 INFO:teuthology.orchestra.run.smithi116.stdout:checking for linux/btrfs.h... yes 2024-02-18T11:49:48.726 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/btrfs_tree.h usability... yes 2024-02-18T11:49:48.735 INFO:teuthology.orchestra.run.smithi116.stdout:checking linux/btrfs_tree.h presence... yes 2024-02-18T11:49:48.735 INFO:teuthology.orchestra.run.smithi116.stdout:checking for linux/btrfs_tree.h... yes 2024-02-18T11:49:48.755 INFO:teuthology.orchestra.run.smithi116.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-02-18T11:49:48.771 INFO:teuthology.orchestra.run.smithi116.stdout:configure: creating ./config.status 2024-02-18T11:49:49.062 INFO:teuthology.orchestra.run.smithi116.stdout:config.status: creating include/builddefs 2024-02-18T11:49:49.072 INFO:teuthology.orchestra.run.smithi116.stdout:config.status: creating include/config.h 2024-02-18T11:49:49.080 INFO:teuthology.orchestra.run.smithi116.stdout:config.status: executing libtool commands 2024-02-18T11:49:49.117 INFO:teuthology.orchestra.run.smithi116.stdout:Building include 2024-02-18T11:49:49.120 INFO:teuthology.orchestra.run.smithi116.stdout:Building lib 2024-02-18T11:49:49.123 INFO:teuthology.orchestra.run.smithi116.stdout: [LTDEP] 2024-02-18T11:49:49.257 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dataascii.lo 2024-02-18T11:49:49.362 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] databin.lo 2024-02-18T11:49:49.496 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] datapid.lo 2024-02-18T11:49:49.589 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] file_lock.lo 2024-02-18T11:49:49.724 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] forker.lo 2024-02-18T11:49:49.845 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] pattern.lo 2024-02-18T11:49:49.953 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] open_flags.lo 2024-02-18T11:49:50.079 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] random_range.lo 2024-02-18T11:49:50.282 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] string_to_tokens.lo 2024-02-18T11:49:50.374 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] str_to_bytes.lo 2024-02-18T11:49:50.493 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] tlibio.lo 2024-02-18T11:49:50.709 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] write_log.lo 2024-02-18T11:49:50.924 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] random.lo 2024-02-18T11:49:51.020 INFO:teuthology.orchestra.run.smithi116.stdout: [LD] 2024-02-18T11:49:51.099 INFO:teuthology.orchestra.run.smithi116.stdout:Building ltp 2024-02-18T11:49:51.101 INFO:teuthology.orchestra.run.smithi116.stdout: [DEP] 2024-02-18T11:49:51.239 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] doio 2024-02-18T11:49:51.841 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fsstress 2024-02-18T11:49:53.140 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fsx 2024-02-18T11:49:53.806 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] growfiles 2024-02-18T11:49:54.260 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] iogen 2024-02-18T11:49:54.597 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-stress 2024-02-18T11:49:54.978 INFO:teuthology.orchestra.run.smithi116.stdout:Building src 2024-02-18T11:49:54.983 INFO:teuthology.orchestra.run.smithi116.stdout: [DEP] 2024-02-18T11:49:56.682 INFO:teuthology.orchestra.run.smithi116.stdout: [DEP] 2024-02-18T11:49:56.712 INFO:teuthology.orchestra.run.smithi116.stdout: [DEP] 2024-02-18T11:49:56.720 INFO:teuthology.orchestra.run.smithi116.stdout: [DEP] 2024-02-18T11:49:56.928 INFO:teuthology.orchestra.run.smithi116.stdout: [DEP] 2024-02-18T11:49:56.949 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dirstress 2024-02-18T11:49:57.153 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fill 2024-02-18T11:49:57.284 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fill2 2024-02-18T11:49:57.484 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] getpagesize 2024-02-18T11:49:57.598 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] holes 2024-02-18T11:49:57.765 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] lstat64 2024-02-18T11:49:57.921 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] nametest 2024-02-18T11:49:58.128 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] permname 2024-02-18T11:49:58.300 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] randholes 2024-02-18T11:49:58.518 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] runas 2024-02-18T11:49:58.666 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] truncfile 2024-02-18T11:49:58.819 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] usemem 2024-02-18T11:49:58.945 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] mmapcat 2024-02-18T11:49:59.071 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] append_reader 2024-02-18T11:49:59.200 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] append_writer 2024-02-18T11:49:59.328 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dirperf 2024-02-18T11:49:59.525 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] metaperf 2024-02-18T11:49:59.763 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] devzero 2024-02-18T11:49:59.913 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] feature 2024-02-18T11:50:00.096 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] alloc 2024-02-18T11:50:00.279 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fault 2024-02-18T11:50:00.420 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fstest 2024-02-18T11:50:00.626 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_access_root 2024-02-18T11:50:00.755 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] godown 2024-02-18T11:50:00.900 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] resvtest 2024-02-18T11:50:01.057 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] writemod 2024-02-18T11:50:01.179 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] writev_on_pagefault 2024-02-18T11:50:01.313 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] makeextents 2024-02-18T11:50:01.465 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] itrash 2024-02-18T11:50:01.593 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] rename 2024-02-18T11:50:01.710 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] multi_open_unlink 2024-02-18T11:50:01.850 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] unwritten_sync 2024-02-18T11:50:02.004 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] genhashnames 2024-02-18T11:50:02.169 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_holes 2024-02-18T11:50:02.293 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_writev 2024-02-18T11:50:02.420 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_truncate_cmtime 2024-02-18T11:50:02.554 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dirhash_collide 2024-02-18T11:50:02.709 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_rename_overwrite 2024-02-18T11:50:02.828 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] holetest 2024-02-18T11:50:03.025 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_truncate_self 2024-02-18T11:50:03.144 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] af_unix 2024-02-18T11:50:03.272 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_stale_pmd 2024-02-18T11:50:03.401 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_cow_race 2024-02-18T11:50:03.542 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_fallocate 2024-02-18T11:50:03.670 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fsync-err 2024-02-18T11:50:03.827 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_write_ro 2024-02-18T11:50:03.956 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_ext4_dax_journal_corruption 2024-02-18T11:50:04.093 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_ext4_dax_inline_corruption 2024-02-18T11:50:04.224 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_ofd_locks 2024-02-18T11:50:04.393 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_collision 2024-02-18T11:50:04.562 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] mmap-write-concurrent 2024-02-18T11:50:04.562 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:50:04.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:50:04.707 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_get_file_time 2024-02-18T11:50:04.832 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_create_short_dirs 2024-02-18T11:50:04.977 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_create_long_dirs 2024-02-18T11:50:05.120 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_enospc 2024-02-18T11:50:05.289 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_writev_overlap 2024-02-18T11:50:05.436 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] checkpoint_journal 2024-02-18T11:50:05.568 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] mmap-rw-fault 2024-02-18T11:50:05.726 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] allocstale 2024-02-18T11:50:05.867 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_cow_memory_failure 2024-02-18T11:50:06.012 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fake-dump-rootino 2024-02-18T11:50:06.138 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dio-buf-fault 2024-02-18T11:50:06.267 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] rewinddir-test 2024-02-18T11:50:06.411 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] readdir-while-renames 2024-02-18T11:50:06.549 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] xfsctl 2024-02-18T11:50:06.710 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] bstat 2024-02-18T11:50:06.888 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mtab 2024-02-18T11:50:07.046 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] getdevicesize 2024-02-18T11:50:07.167 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] preallo_rw_pattern_reader 2024-02-18T11:50:07.299 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] preallo_rw_pattern_writer 2024-02-18T11:50:07.433 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] ftrunc 2024-02-18T11:50:07.557 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] trunc 2024-02-18T11:50:07.695 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fs_perms 2024-02-18T11:50:07.834 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] testx 2024-02-18T11:50:07.944 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] looptest 2024-02-18T11:50:08.110 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] locktest 2024-02-18T11:50:08.380 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] unwritten_mmap 2024-02-18T11:50:08.518 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] bulkstat_unlink_test 2024-02-18T11:50:08.691 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] deduperace 2024-02-18T11:50:08.858 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] bulkstat_unlink_test_modified 2024-02-18T11:50:09.027 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_dir_offset 2024-02-18T11:50:09.152 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_futimens 2024-02-18T11:50:09.269 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_immutable 2024-02-18T11:50:09.826 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] stale_handle 2024-02-18T11:50:09.970 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] pwrite_mmap_blocked 2024-02-18T11:50:10.100 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_dir_offset2 2024-02-18T11:50:10.265 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] seek_sanity_test 2024-02-18T11:50:10.660 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] seek_copy_test 2024-02-18T11:50:10.811 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_readdir_1 2024-02-18T11:50:10.936 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_readdir_2 2024-02-18T11:50:11.063 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fsync-tester 2024-02-18T11:50:11.283 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] nsexec 2024-02-18T11:50:11.437 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] cloner 2024-02-18T11:50:11.590 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] renameat2 2024-02-18T11:50:11.732 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_getcwd 2024-02-18T11:50:11.867 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] e4compact 2024-02-18T11:50:12.041 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] test-nextquota 2024-02-18T11:50:12.187 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] punch-alternating 2024-02-18T11:50:12.332 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] attr-list-by-handle-cursor-test 2024-02-18T11:50:12.475 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] listxattr 2024-02-18T11:50:12.600 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dio-interleaved 2024-02-18T11:50:12.738 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_dir_type 2024-02-18T11:50:12.870 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dio-invalidate-cache 2024-02-18T11:50:13.045 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] stat_test 2024-02-18T11:50:13.303 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_encrypted_d_revalidate 2024-02-18T11:50:13.443 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] attr_replace_test 2024-02-18T11:50:13.582 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] swapon 2024-02-18T11:50:13.723 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] mkswap 2024-02-18T11:50:13.851 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_attr_corruption 2024-02-18T11:50:13.977 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_open_tmpfiles 2024-02-18T11:50:14.132 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fscrypt-crypt-util 2024-02-18T11:50:14.733 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] bulkstat_null_ocount 2024-02-18T11:50:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] splice-test 2024-02-18T11:50:15.022 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] chprojid_fail 2024-02-18T11:50:15.159 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] detached_mounts_propagation 2024-02-18T11:50:15.318 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] ext4_resize 2024-02-18T11:50:15.447 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_readdir_3 2024-02-18T11:50:15.616 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] splice2pipe 2024-02-18T11:50:15.757 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] uuid_ioctl 2024-02-18T11:50:15.888 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_snapshot_deleted_subvolume 2024-02-18T11:50:16.029 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fiemap-tester 2024-02-18T11:50:16.264 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_stripealign 2024-02-18T11:50:16.396 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] open_by_handle 2024-02-18T11:50:16.615 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] dbtest 2024-02-18T11:50:16.830 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_mmap_dio 2024-02-18T11:50:16.967 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] xfsfind 2024-02-18T11:50:17.134 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] t_reflink_read_race 2024-02-18T11:50:17.299 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] fssum 2024-02-18T11:50:17.699 INFO:teuthology.orchestra.run.smithi116.stdout:Building log-writes 2024-02-18T11:50:17.704 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] replay-log 2024-02-18T11:50:17.982 INFO:teuthology.orchestra.run.smithi116.stdout:Building perf 2024-02-18T11:50:17.985 INFO:teuthology.orchestra.run.smithi116.stdout:Building aio-dio-regress 2024-02-18T11:50:17.991 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aiocp 2024-02-18T11:50:18.188 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-append-write-fallocate-race 2024-02-18T11:50:18.331 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-append-write-read-race 2024-02-18T11:50:18.480 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-cow-race 2024-02-18T11:50:18.611 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-cycle-write 2024-02-18T11:50:18.768 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-eof-race 2024-02-18T11:50:18.925 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-extend-stat 2024-02-18T11:50:19.064 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-fcntl-race 2024-02-18T11:50:19.204 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-hole-filling-race 2024-02-18T11:50:19.329 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-invalidate-failure 2024-02-18T11:50:19.465 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-invalidate-readahead 2024-02-18T11:50:19.606 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aiodio_sparse2 2024-02-18T11:50:19.795 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-subblock-eof-read 2024-02-18T11:50:19.915 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-dio-write-verify 2024-02-18T11:50:20.085 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-02-18T11:50:20.198 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-02-18T11:50:20.309 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] aio-last-ref-held-by-io 2024-02-18T11:50:20.459 INFO:teuthology.orchestra.run.smithi116.stdout:Building vfs 2024-02-18T11:50:20.464 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] vfstest 2024-02-18T11:50:23.763 INFO:teuthology.orchestra.run.smithi116.stdout: [CC] mount-idmapped 2024-02-18T11:50:24.109 INFO:teuthology.orchestra.run.smithi116.stdout:Building m4 2024-02-18T11:50:24.112 INFO:teuthology.orchestra.run.smithi116.stdout:Building common 2024-02-18T11:50:24.116 INFO:teuthology.orchestra.run.smithi116.stdout:Building tests 2024-02-18T11:50:24.121 INFO:teuthology.orchestra.run.smithi116.stdout:Building btrfs 2024-02-18T11:50:24.127 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/btrfs/group.list 2024-02-18T11:50:24.186 INFO:teuthology.orchestra.run.smithi116.stdout:Building ceph 2024-02-18T11:50:24.190 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ceph/group.list 2024-02-18T11:50:24.204 INFO:teuthology.orchestra.run.smithi116.stdout:Building cifs 2024-02-18T11:50:24.209 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/cifs/group.list 2024-02-18T11:50:24.221 INFO:teuthology.orchestra.run.smithi116.stdout:Building ext4 2024-02-18T11:50:24.226 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ext4/group.list 2024-02-18T11:50:24.264 INFO:teuthology.orchestra.run.smithi116.stdout:Building f2fs 2024-02-18T11:50:24.268 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/f2fs/group.list 2024-02-18T11:50:24.285 INFO:teuthology.orchestra.run.smithi116.stdout:Building generic 2024-02-18T11:50:24.291 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/generic/group.list 2024-02-18T11:50:24.387 INFO:teuthology.orchestra.run.smithi116.stdout:Building nfs 2024-02-18T11:50:24.390 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/nfs/group.list 2024-02-18T11:50:24.404 INFO:teuthology.orchestra.run.smithi116.stdout:Building ocfs2 2024-02-18T11:50:24.409 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ocfs2/group.list 2024-02-18T11:50:24.422 INFO:teuthology.orchestra.run.smithi116.stdout:Building overlay 2024-02-18T11:50:24.427 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/overlay/group.list 2024-02-18T11:50:24.466 INFO:teuthology.orchestra.run.smithi116.stdout:Building perf 2024-02-18T11:50:24.471 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/perf/group.list 2024-02-18T11:50:24.482 INFO:teuthology.orchestra.run.smithi116.stdout:Building selftest 2024-02-18T11:50:24.487 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/selftest/group.list 2024-02-18T11:50:24.499 INFO:teuthology.orchestra.run.smithi116.stdout:Building shared 2024-02-18T11:50:24.504 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/shared/group.list 2024-02-18T11:50:24.521 INFO:teuthology.orchestra.run.smithi116.stdout:Building tmpfs 2024-02-18T11:50:24.526 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/tmpfs/group.list 2024-02-18T11:50:24.539 INFO:teuthology.orchestra.run.smithi116.stdout:Building udf 2024-02-18T11:50:24.543 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/udf/group.list 2024-02-18T11:50:24.554 INFO:teuthology.orchestra.run.smithi116.stdout:Building xfs 2024-02-18T11:50:24.561 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/xfs/group.list 2024-02-18T11:50:24.657 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo make install) 2024-02-18T11:50:24.695 INFO:teuthology.orchestra.run.smithi116.stdout:Building include 2024-02-18T11:50:24.698 INFO:teuthology.orchestra.run.smithi116.stdout:Building lib 2024-02-18T11:50:24.702 INFO:teuthology.orchestra.run.smithi116.stdout:Building ltp 2024-02-18T11:50:24.706 INFO:teuthology.orchestra.run.smithi116.stdout:Building src 2024-02-18T11:50:24.739 INFO:teuthology.orchestra.run.smithi116.stdout:Building log-writes 2024-02-18T11:50:24.742 INFO:teuthology.orchestra.run.smithi116.stdout:Building perf 2024-02-18T11:50:24.745 INFO:teuthology.orchestra.run.smithi116.stdout:Building aio-dio-regress 2024-02-18T11:50:24.749 INFO:teuthology.orchestra.run.smithi116.stdout:Building vfs 2024-02-18T11:50:24.753 INFO:teuthology.orchestra.run.smithi116.stdout:Building m4 2024-02-18T11:50:24.756 INFO:teuthology.orchestra.run.smithi116.stdout:Building common 2024-02-18T11:50:24.760 INFO:teuthology.orchestra.run.smithi116.stdout:Building tests 2024-02-18T11:50:24.765 INFO:teuthology.orchestra.run.smithi116.stdout:Building btrfs 2024-02-18T11:50:24.770 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/btrfs/group.list 2024-02-18T11:50:24.828 INFO:teuthology.orchestra.run.smithi116.stdout:Building ceph 2024-02-18T11:50:24.833 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ceph/group.list 2024-02-18T11:50:24.846 INFO:teuthology.orchestra.run.smithi116.stdout:Building cifs 2024-02-18T11:50:24.851 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/cifs/group.list 2024-02-18T11:50:24.863 INFO:teuthology.orchestra.run.smithi116.stdout:Building ext4 2024-02-18T11:50:24.868 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ext4/group.list 2024-02-18T11:50:24.905 INFO:teuthology.orchestra.run.smithi116.stdout:Building f2fs 2024-02-18T11:50:24.910 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/f2fs/group.list 2024-02-18T11:50:24.927 INFO:teuthology.orchestra.run.smithi116.stdout:Building generic 2024-02-18T11:50:24.933 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/generic/group.list 2024-02-18T11:50:25.025 INFO:teuthology.orchestra.run.smithi116.stdout:Building nfs 2024-02-18T11:50:25.030 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/nfs/group.list 2024-02-18T11:50:25.043 INFO:teuthology.orchestra.run.smithi116.stdout:Building ocfs2 2024-02-18T11:50:25.048 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ocfs2/group.list 2024-02-18T11:50:25.061 INFO:teuthology.orchestra.run.smithi116.stdout:Building overlay 2024-02-18T11:50:25.066 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/overlay/group.list 2024-02-18T11:50:25.106 INFO:teuthology.orchestra.run.smithi116.stdout:Building perf 2024-02-18T11:50:25.110 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/perf/group.list 2024-02-18T11:50:25.122 INFO:teuthology.orchestra.run.smithi116.stdout:Building selftest 2024-02-18T11:50:25.127 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/selftest/group.list 2024-02-18T11:50:25.139 INFO:teuthology.orchestra.run.smithi116.stdout:Building shared 2024-02-18T11:50:25.144 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/shared/group.list 2024-02-18T11:50:25.160 INFO:teuthology.orchestra.run.smithi116.stdout:Building tmpfs 2024-02-18T11:50:25.165 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/tmpfs/group.list 2024-02-18T11:50:25.179 INFO:teuthology.orchestra.run.smithi116.stdout:Building udf 2024-02-18T11:50:25.183 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/udf/group.list 2024-02-18T11:50:25.194 INFO:teuthology.orchestra.run.smithi116.stdout:Building xfs 2024-02-18T11:50:25.200 INFO:teuthology.orchestra.run.smithi116.stdout: [GROUP] /tmp/tmp.hC7pXm5hUExfstests-dev/tests/xfs/group.list 2024-02-18T11:50:25.293 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-02-18T11:50:25.294 INFO:teuthology.orchestra.run.smithi116.stdout:gmake[1]: Nothing to be done for 'install'. 2024-02-18T11:50:25.294 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-02-18T11:50:25.296 INFO:teuthology.orchestra.run.smithi116.stdout:gmake[1]: Nothing to be done for 'install'. 2024-02-18T11:50:25.296 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-02-18T11:50:25.299 INFO:teuthology.orchestra.run.smithi116.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-02-18T11:50:25.307 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-02-18T11:50:25.373 INFO:teuthology.orchestra.run.smithi116.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-02-18T11:50:25.382 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-02-18T11:50:25.398 INFO:teuthology.orchestra.run.smithi116.stdout:Building log-writes 2024-02-18T11:50:25.402 INFO:teuthology.orchestra.run.smithi116.stdout:Building perf 2024-02-18T11:50:25.405 INFO:teuthology.orchestra.run.smithi116.stdout:Building aio-dio-regress 2024-02-18T11:50:25.409 INFO:teuthology.orchestra.run.smithi116.stdout:Building vfs 2024-02-18T11:50:25.412 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake -C log-writes install 2024-02-18T11:50:25.413 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-02-18T11:50:25.421 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-02-18T11:50:25.430 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake -C perf install 2024-02-18T11:50:25.431 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-02-18T11:50:25.439 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-02-18T11:50:25.460 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-02-18T11:50:25.461 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-02-18T11:50:25.469 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-02-18T11:50:25.516 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake -C vfs install 2024-02-18T11:50:25.518 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-02-18T11:50:25.526 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-02-18T11:50:25.537 INFO:teuthology.orchestra.run.smithi116.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-02-18T11:50:25.545 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-02-18T11:50:26.656 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-02-18T11:50:26.746 INFO:teuthology.orchestra.run.smithi116.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-02-18T11:50:26.770 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-02-18T11:50:26.771 INFO:teuthology.orchestra.run.smithi116.stdout:gmake[1]: Nothing to be done for 'install'. 2024-02-18T11:50:26.771 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-02-18T11:50:26.773 INFO:teuthology.orchestra.run.smithi116.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-02-18T11:50:26.781 INFO:teuthology.orchestra.run.smithi116.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-02-18T11:50:26.900 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-02-18T11:50:26.901 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/btrfs/ install 2024-02-18T11:50:26.905 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-02-18T11:50:26.913 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 /var/lib/xfstests/tests/btrfs 2024-02-18T11:50:27.663 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-02-18T11:50:27.669 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out /var/lib/xfstests/tests/btrfs 2024-02-18T11:50:28.422 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ceph/ install 2024-02-18T11:50:28.424 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-02-18T11:50:28.432 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-02-18T11:50:28.450 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-02-18T11:50:28.458 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-02-18T11:50:28.475 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/cifs/ install 2024-02-18T11:50:28.477 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-02-18T11:50:28.485 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-02-18T11:50:28.493 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-02-18T11:50:28.502 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-02-18T11:50:28.510 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ext4/ install 2024-02-18T11:50:28.512 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-02-18T11:50:28.520 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-02-18T11:50:28.687 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-02-18T11:50:28.695 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-02-18T11:50:28.872 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/f2fs/ install 2024-02-18T11:50:28.873 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-02-18T11:50:28.881 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-02-18T11:50:28.892 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-02-18T11:50:28.900 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-02-18T11:50:28.911 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/generic/ install 2024-02-18T11:50:28.916 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-02-18T11:50:28.924 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-02-18T11:50:30.721 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-02-18T11:50:30.729 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-02-18T11:50:32.553 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/nfs/ install 2024-02-18T11:50:32.554 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-02-18T11:50:32.562 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-02-18T11:50:32.570 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-02-18T11:50:32.579 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-02-18T11:50:32.587 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/ocfs2/ install 2024-02-18T11:50:32.588 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-02-18T11:50:32.597 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-02-18T11:50:32.605 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-02-18T11:50:32.613 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-02-18T11:50:32.621 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/overlay/ install 2024-02-18T11:50:32.623 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-02-18T11:50:32.631 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-02-18T11:50:32.889 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-02-18T11:50:32.897 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-02-18T11:50:33.155 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/perf/ install 2024-02-18T11:50:33.156 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-02-18T11:50:33.164 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-02-18T11:50:33.172 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-02-18T11:50:33.181 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-02-18T11:50:33.189 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/selftest/ install 2024-02-18T11:50:33.190 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-02-18T11:50:33.198 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-02-18T11:50:33.218 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-02-18T11:50:33.227 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-02-18T11:50:33.247 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/shared/ install 2024-02-18T11:50:33.248 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-02-18T11:50:33.257 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-02-18T11:50:33.270 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-02-18T11:50:33.278 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-02-18T11:50:33.291 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/tmpfs/ install 2024-02-18T11:50:33.292 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-02-18T11:50:33.300 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-02-18T11:50:33.308 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-02-18T11:50:33.317 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-02-18T11:50:33.325 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/udf/ install 2024-02-18T11:50:33.326 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-02-18T11:50:33.334 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-02-18T11:50:33.342 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-02-18T11:50:33.351 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-02-18T11:50:33.359 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.hC7pXm5hUExfstests-dev/tests/xfs/ install 2024-02-18T11:50:33.364 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-02-18T11:50:33.372 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 602 603 604 605 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-02-18T11:50:34.626 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:50:34.630 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:50:35.123 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-02-18T11:50:35.132 INFO:teuthology.orchestra.run.smithi116.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 602.out 603.out 604.out 605.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-02-18T11:50:36.846 INFO:teuthology.orchestra.run.smithi116.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-02-18T11:50:36.853 INFO:teuthology.orchestra.run.smithi116.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-02-18T11:50:36.861 INFO:teuthology.orchestra.run.smithi116.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-02-18T11:50:36.869 INFO:teuthology.orchestra.run.smithi116.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-02-18T11:50:36.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T11:50:36.880 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec ls tests/generic) 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:001 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:001.out 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:002 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:002.out 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:003 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:003.out 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:004 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:004.out 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:005 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:005.out 2024-02-18T11:50:36.897 INFO:teuthology.orchestra.run.smithi116.stdout:006 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:006.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:007 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:007.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:008 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:008.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:009 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:009.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:010 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:010.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:011 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:011.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:012 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:012.out 2024-02-18T11:50:36.898 INFO:teuthology.orchestra.run.smithi116.stdout:013 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:013.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:014 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:014.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:015 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:015.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:016 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:016.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:017 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:017.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:018 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:018.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:019 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:019.out 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:020 2024-02-18T11:50:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:020.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:021 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:021.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:022 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:022.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:023 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:023.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:024 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:024.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:025 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:025.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:026 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:026.out 2024-02-18T11:50:36.900 INFO:teuthology.orchestra.run.smithi116.stdout:027 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:027.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:028 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:028.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:029 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:029.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:030 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:030.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:031 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:031.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:032 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:032.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:033 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:033.out 2024-02-18T11:50:36.901 INFO:teuthology.orchestra.run.smithi116.stdout:034 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:034.out 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:035 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:035.cfg 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:035.out.default 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:035.out.nfs 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:036 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:036.out 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:037 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:037.out 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:038 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:038.out 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:039 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:039.out 2024-02-18T11:50:36.902 INFO:teuthology.orchestra.run.smithi116.stdout:040 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:040.out 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:041 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:041.out 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:042 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:042.out 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:043 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:043.out 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:044 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:044.out 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:045 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:045.out 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:046 2024-02-18T11:50:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:046.out 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:047 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:047.out 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:048 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:048.out 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:049 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:049.out 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:050 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:050.cfg 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:050.out.default 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:050.out.nojournal 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:050.out.xfsquota 2024-02-18T11:50:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:051 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:051.out 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:052 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:052.out 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:053 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:053.out 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:054 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:054.out 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:055 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:055.out 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:056 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:056.out 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:057 2024-02-18T11:50:36.905 INFO:teuthology.orchestra.run.smithi116.stdout:057.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:058 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:058.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:059 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:059.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:060 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:060.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:061 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:061.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:062 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:062.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:063 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:063.out 2024-02-18T11:50:36.906 INFO:teuthology.orchestra.run.smithi116.stdout:064 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:064.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:065 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:065.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:066 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:066.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:067 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:067.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:068 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:068.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:069 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:069.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:070 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:070.out 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:071 2024-02-18T11:50:36.907 INFO:teuthology.orchestra.run.smithi116.stdout:071.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:072 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:072.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:073 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:073.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:074 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:074.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:075 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:075.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:076 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:076.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:077 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:077.out 2024-02-18T11:50:36.908 INFO:teuthology.orchestra.run.smithi116.stdout:078 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:078.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:079 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:079.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:080 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:080.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:081 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:081.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:082 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:082.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:083 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:083.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:084 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:084.out 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:085 2024-02-18T11:50:36.909 INFO:teuthology.orchestra.run.smithi116.stdout:085.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:086 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:086.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:087 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:087.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:088 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:088.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:089 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:089.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:090 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:090.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:091 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:091.out 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:092 2024-02-18T11:50:36.910 INFO:teuthology.orchestra.run.smithi116.stdout:092.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:093 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:093.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:094 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:094.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:095 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:095.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:096 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:096.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:097 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:097.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:098 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:098.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:099 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:099.out 2024-02-18T11:50:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:100 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:100.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:101 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:101.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:102 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:102.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:103 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:103.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:104 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:104.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:105 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:105.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:106 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:106.out 2024-02-18T11:50:36.912 INFO:teuthology.orchestra.run.smithi116.stdout:107 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:107.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:108 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:108.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:109 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:109.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:110 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:110.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:111 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:111.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:112 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:112.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:113 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:113.out 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:114 2024-02-18T11:50:36.913 INFO:teuthology.orchestra.run.smithi116.stdout:114.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:115 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:115.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:116 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:116.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:117 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:117.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:118 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:118.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:119 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:119.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:120 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:120.out 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:121 2024-02-18T11:50:36.914 INFO:teuthology.orchestra.run.smithi116.stdout:121.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:122 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:122.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:123 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:123.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:124 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:124.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:125 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:125.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:126 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:126.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:127 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:127.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:128 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:128.out 2024-02-18T11:50:36.915 INFO:teuthology.orchestra.run.smithi116.stdout:129 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:129.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:130 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:130.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:131 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:131.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:132 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:132.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:133 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:133.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:134 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:134.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:135 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:135.out 2024-02-18T11:50:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:136 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:136.out 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:137 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:137.out 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:138 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:138.out 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:139 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:139.out 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:140 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:140.out 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:141 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:141.out 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:142 2024-02-18T11:50:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:142.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:143 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:143.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:144 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:144.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:145 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:145.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:146 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:146.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:147 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:147.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:148 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:148.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:149 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:149.out 2024-02-18T11:50:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:150 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:150.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:151 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:151.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:152 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:152.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:153 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:153.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:154 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:154.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:155 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:155.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:156 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:156.out 2024-02-18T11:50:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:157 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:157.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:158 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:158.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:159 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:159.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:160 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:160.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:161 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:161.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:162 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:162.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:163 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:163.out 2024-02-18T11:50:36.920 INFO:teuthology.orchestra.run.smithi116.stdout:164 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:164.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:165 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:165.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:166 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:166.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:167 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:167.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:168 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:168.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:169 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:169.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:170 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:170.out 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:171 2024-02-18T11:50:36.921 INFO:teuthology.orchestra.run.smithi116.stdout:171.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:172 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:172.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:173 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:173.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:174 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:174.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:175 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:175.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:176 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:176.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:177 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:177.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:178 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:178.out 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:179 2024-02-18T11:50:36.922 INFO:teuthology.orchestra.run.smithi116.stdout:179.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:180 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:180.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:181 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:181.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:182 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:182.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:183 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:183.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:184 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:184.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:185 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:185.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:186 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:186.out 2024-02-18T11:50:36.923 INFO:teuthology.orchestra.run.smithi116.stdout:187 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:187.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:188 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:188.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:189 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:189.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:190 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:190.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:191 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:191.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:192 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:192.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:193 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:193.out 2024-02-18T11:50:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:194 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:194.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:195 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:195.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:196 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:196.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:197 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:197.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:198 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:198.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:199 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:199.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:200 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:200.out 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:201 2024-02-18T11:50:36.925 INFO:teuthology.orchestra.run.smithi116.stdout:201.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:202 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:202.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:203 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:203.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:204 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:204.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:205 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:205.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:206 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:206.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:207 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:207.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:208 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:208.out 2024-02-18T11:50:36.926 INFO:teuthology.orchestra.run.smithi116.stdout:209 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:209.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:210 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:210.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:211 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:211.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:212 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:212.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:213 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:213.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:214 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:214.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:215 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:215.out 2024-02-18T11:50:36.927 INFO:teuthology.orchestra.run.smithi116.stdout:216 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:216.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:217 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:217.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:218 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:218.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:219 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:219.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:220 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:220.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:221 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:221.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:222 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:222.out 2024-02-18T11:50:36.928 INFO:teuthology.orchestra.run.smithi116.stdout:223 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:223.out 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:224 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:224.out 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:225 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:225.out 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:226 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:226.out 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:227 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:227.out 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:228 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:228.out 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:229 2024-02-18T11:50:36.929 INFO:teuthology.orchestra.run.smithi116.stdout:229.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:230 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:230.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:231 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:231.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:232 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:232.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:233 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:233.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:234 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:234.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:235 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:235.out 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:236 2024-02-18T11:50:36.930 INFO:teuthology.orchestra.run.smithi116.stdout:236.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:237 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:237.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:238 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:238.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:239 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:239.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:240 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:240.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:241 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:241.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:242 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:242.out 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:243 2024-02-18T11:50:36.931 INFO:teuthology.orchestra.run.smithi116.stdout:243.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:244 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:244.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:245 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:245.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:246 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:246.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:247 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:247.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:248 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:248.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:249 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:249.out 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:250 2024-02-18T11:50:36.932 INFO:teuthology.orchestra.run.smithi116.stdout:250.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:251 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:251.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:252 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:252.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:253 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:253.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:254 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:254.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:255 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:255.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:256 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:256.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:257 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:257.out 2024-02-18T11:50:36.933 INFO:teuthology.orchestra.run.smithi116.stdout:258 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:258.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:259 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:259.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:260 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:260.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:261 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:261.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:262 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:262.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:263 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:263.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:264 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:264.out 2024-02-18T11:50:36.934 INFO:teuthology.orchestra.run.smithi116.stdout:265 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:265.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:266 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:266.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:267 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:267.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:268 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:268.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:269 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:269.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:270 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:270.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:271 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:271.out 2024-02-18T11:50:36.935 INFO:teuthology.orchestra.run.smithi116.stdout:272 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:272.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:273 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:273.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:274 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:274.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:275 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:275.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:276 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:276.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:277 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:277.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:278 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:278.out 2024-02-18T11:50:36.936 INFO:teuthology.orchestra.run.smithi116.stdout:279 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:279.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:280 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:280.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:281 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:281.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:282 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:282.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:283 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:283.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:284 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:284.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:285 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:285.out 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:286 2024-02-18T11:50:36.937 INFO:teuthology.orchestra.run.smithi116.stdout:286.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:287 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:287.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:288 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:288.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:289 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:289.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:290 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:290.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:291 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:291.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:292 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:292.out 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:293 2024-02-18T11:50:36.938 INFO:teuthology.orchestra.run.smithi116.stdout:293.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:294 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:294.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:295 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:295.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:296 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:296.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:297 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:297.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:298 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:298.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:299 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:299.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:300 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:300.out 2024-02-18T11:50:36.939 INFO:teuthology.orchestra.run.smithi116.stdout:301 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:301.out 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:302 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:302.out 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:303 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:303.out 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:304 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:304.out 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:305 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:305.out 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:306 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:306.out 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:307 2024-02-18T11:50:36.940 INFO:teuthology.orchestra.run.smithi116.stdout:307.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:308 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:308.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:309 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:309.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:310 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:310.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:311 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:311.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:312 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:312.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:313 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:313.out 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:314 2024-02-18T11:50:36.941 INFO:teuthology.orchestra.run.smithi116.stdout:314.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:315 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:315.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:316 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:316.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:317 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:317.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:318 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:318.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:319 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:319.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:320 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:320.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:321 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:321.out 2024-02-18T11:50:36.942 INFO:teuthology.orchestra.run.smithi116.stdout:322 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:322.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:323 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:323.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:324 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:324.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:325 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:325.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:326 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:326.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:327 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:327.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:328 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:328.out 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:329 2024-02-18T11:50:36.943 INFO:teuthology.orchestra.run.smithi116.stdout:329.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:330 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:330.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:331 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:331.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:332 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:332.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:333 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:333.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:334 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:334.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:335 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:335.out 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:336 2024-02-18T11:50:36.944 INFO:teuthology.orchestra.run.smithi116.stdout:336.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:337 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:337.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:338 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:338.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:339 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:339.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:340 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:340.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:341 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:341.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:342 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:342.out 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:343 2024-02-18T11:50:36.945 INFO:teuthology.orchestra.run.smithi116.stdout:343.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:344 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:344.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:345 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:345.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:346 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:346.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:347 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:347.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:348 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:348.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:349 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:349.out 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:350 2024-02-18T11:50:36.946 INFO:teuthology.orchestra.run.smithi116.stdout:350.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:351 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:351.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:352 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:352.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:353 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:353.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:354 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:354.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:355 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:355.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:356 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:356.out 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:357 2024-02-18T11:50:36.947 INFO:teuthology.orchestra.run.smithi116.stdout:357.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:358 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:358.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:359 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:359.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:360 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:360.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:361 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:361.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:362 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:362.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:363 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:363.out 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:364 2024-02-18T11:50:36.948 INFO:teuthology.orchestra.run.smithi116.stdout:364.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:365 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:365.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:366 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:366.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:367 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:367.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:368 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:368.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:369 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:369.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:370 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:370.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:371 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:371.out 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:372 2024-02-18T11:50:36.949 INFO:teuthology.orchestra.run.smithi116.stdout:372.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:373 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:373.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:374 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:374.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:375 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:375.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:376 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:376.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:377 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:377.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:378 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:378.out 2024-02-18T11:50:36.950 INFO:teuthology.orchestra.run.smithi116.stdout:379 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:379.out 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:380 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:380.out 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:381 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:381.out 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:382 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:382.out 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:383 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:383.out 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:384 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:384.out 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:385 2024-02-18T11:50:36.951 INFO:teuthology.orchestra.run.smithi116.stdout:385.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:386 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:386.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:387 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:387.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:388 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:388.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:389 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:389.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:390 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:390.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:391 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:391.out 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:392 2024-02-18T11:50:36.952 INFO:teuthology.orchestra.run.smithi116.stdout:392.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:393 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:393.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:394 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:394.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:395 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:395.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:396 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:396.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:397 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:397.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:398 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:398.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:399 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:399.out 2024-02-18T11:50:36.953 INFO:teuthology.orchestra.run.smithi116.stdout:400 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:400.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:401 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:401.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:402 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:402.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:403 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:403.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:404 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:404.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:405 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:405.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:406 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:406.out 2024-02-18T11:50:36.954 INFO:teuthology.orchestra.run.smithi116.stdout:407 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:407.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:408 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:408.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:409 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:409.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:410 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:410.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:411 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:411.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:412 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:412.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:413 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:413.out 2024-02-18T11:50:36.955 INFO:teuthology.orchestra.run.smithi116.stdout:414 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:414.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:415 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:415.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:416 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:416.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:417 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:417.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:418 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:418.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:419 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:419.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:420 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:420.out 2024-02-18T11:50:36.956 INFO:teuthology.orchestra.run.smithi116.stdout:421 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:421.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:422 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:422.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:423 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:423.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:424 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:424.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:425 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:425.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:426 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:426.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:427 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:427.out 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:428 2024-02-18T11:50:36.957 INFO:teuthology.orchestra.run.smithi116.stdout:428.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:429 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:429.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:430 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:430.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:431 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:431.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:432 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:432.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:433 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:433.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:434 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:434.out 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:435 2024-02-18T11:50:36.958 INFO:teuthology.orchestra.run.smithi116.stdout:435.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:436 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:436.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:437 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:437.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:438 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:438.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:439 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:439.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:440 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:440.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:441 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:441.out 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:442 2024-02-18T11:50:36.959 INFO:teuthology.orchestra.run.smithi116.stdout:442.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:443 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:443.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:444 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:444.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:445 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:445.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:446 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:446.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:447 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:447.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:448 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:448.out 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:449 2024-02-18T11:50:36.960 INFO:teuthology.orchestra.run.smithi116.stdout:449.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:450 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:450.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:451 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:451.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:452 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:452.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:453 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:453.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:454 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:454.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:455 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:455.out 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:456 2024-02-18T11:50:36.961 INFO:teuthology.orchestra.run.smithi116.stdout:456.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:457 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:457.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:458 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:458.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:459 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:459.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:460 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:460.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:461 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:461.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:462 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:462.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:463 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:463.out 2024-02-18T11:50:36.962 INFO:teuthology.orchestra.run.smithi116.stdout:464 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:464.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:465 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:465.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:466 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:466.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:467 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:467.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:468 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:468.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:469 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:469.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:470 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:470.out 2024-02-18T11:50:36.963 INFO:teuthology.orchestra.run.smithi116.stdout:471 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:471.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:472 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:472.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:473 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:473.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:474 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:474.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:475 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:475.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:476 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:476.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:477 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:477.out 2024-02-18T11:50:36.964 INFO:teuthology.orchestra.run.smithi116.stdout:478 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:478.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:479 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:479.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:480 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:480.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:481 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:481.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:482 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:482.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:483 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:483.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:484 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:484.out 2024-02-18T11:50:36.965 INFO:teuthology.orchestra.run.smithi116.stdout:485 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:485.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:486 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:486.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:487 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:487.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:488 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:488.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:489 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:489.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:490 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:490.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:491 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:491.out 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:492 2024-02-18T11:50:36.966 INFO:teuthology.orchestra.run.smithi116.stdout:492.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:493 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:493.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:494 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:494.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:495 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:495.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:496 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:496.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:497 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:497.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:498 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:498.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:499 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:499.out 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:500 2024-02-18T11:50:36.967 INFO:teuthology.orchestra.run.smithi116.stdout:500.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:501 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:501.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:502 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:502.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:503 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:503.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:504 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:504.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:505 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:505.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:506 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:506.out 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:507 2024-02-18T11:50:36.968 INFO:teuthology.orchestra.run.smithi116.stdout:507.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:508 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:508.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:509 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:509.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:510 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:510.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:511 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:511.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:512 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:512.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:513 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:513.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:514 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:514.out 2024-02-18T11:50:36.969 INFO:teuthology.orchestra.run.smithi116.stdout:515 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:515.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:516 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:516.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:517 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:517.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:518 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:518.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:519 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:519.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:520 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:520.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:521 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:521.out 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:522 2024-02-18T11:50:36.970 INFO:teuthology.orchestra.run.smithi116.stdout:522.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:523 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:523.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:524 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:524.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:525 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:525.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:526 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:526.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:527 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:527.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:528 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:528.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:529 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:529.out 2024-02-18T11:50:36.971 INFO:teuthology.orchestra.run.smithi116.stdout:530 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:530.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:531 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:531.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:532 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:532.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:533 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:533.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:534 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:534.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:535 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:535.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:536 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:536.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:537 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:537.out 2024-02-18T11:50:36.972 INFO:teuthology.orchestra.run.smithi116.stdout:538 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:538.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:539 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:539.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:540 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:540.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:541 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:541.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:542 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:542.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:543 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:543.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:544 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:544.out 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:545 2024-02-18T11:50:36.973 INFO:teuthology.orchestra.run.smithi116.stdout:545.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:546 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:546.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:547 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:547.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:548 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:548.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:549 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:549.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:550 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:550.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:551 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:551.out 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:552 2024-02-18T11:50:36.974 INFO:teuthology.orchestra.run.smithi116.stdout:552.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:553 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:553.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:554 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:554.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:555 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:555.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:556 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:556.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:557 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:557.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:558 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:558.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:559 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:559.out 2024-02-18T11:50:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:560 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:560.out 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:561 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:561.out 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:562 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:562.out 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:563 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:563.out 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:564 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:564.out 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:565 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:565.out 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:566 2024-02-18T11:50:36.976 INFO:teuthology.orchestra.run.smithi116.stdout:566.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:567 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:567.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:568 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:568.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:569 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:569.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:570 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:570.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:571 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:571.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:572 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:572.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:573 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:573.out 2024-02-18T11:50:36.977 INFO:teuthology.orchestra.run.smithi116.stdout:574 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:574.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:575 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:575.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:576 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:576.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:577 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:577.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:578 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:578.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:579 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:579.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:580 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:580.out 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:581 2024-02-18T11:50:36.978 INFO:teuthology.orchestra.run.smithi116.stdout:581.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:582 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:582.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:583 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:583.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:584 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:584.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:585 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:585.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:586 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:586.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:587 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:587.out 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:588 2024-02-18T11:50:36.979 INFO:teuthology.orchestra.run.smithi116.stdout:588.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:589 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:589.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:590 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:590.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:591 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:591.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:592 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:592.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:593 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:593.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:594 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:594.out 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:595 2024-02-18T11:50:36.980 INFO:teuthology.orchestra.run.smithi116.stdout:595.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:596 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:596.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:597 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:597.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:598 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:598.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:599 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:599.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:600 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:600.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:601 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:601.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:602 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:602.out 2024-02-18T11:50:36.981 INFO:teuthology.orchestra.run.smithi116.stdout:603 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:603.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:604 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:604.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:605 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:605.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:606 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:606.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:607 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:607.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:608 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:608.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:609 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:609.out 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:610 2024-02-18T11:50:36.982 INFO:teuthology.orchestra.run.smithi116.stdout:610.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:611 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:611.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:612 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:612.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:613 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:613.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:614 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:614.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:615 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:615.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:616 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:616.out 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:617 2024-02-18T11:50:36.983 INFO:teuthology.orchestra.run.smithi116.stdout:617.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:618 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:618.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:619 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:619.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:620 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:620.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:621 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:621.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:622 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:622.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:623 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:623.out 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:624 2024-02-18T11:50:36.984 INFO:teuthology.orchestra.run.smithi116.stdout:624.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:625 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:625.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:626 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:626.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:627 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:627.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:628 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:628.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:629 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:629.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:630 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:630.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:631 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:631.out 2024-02-18T11:50:36.985 INFO:teuthology.orchestra.run.smithi116.stdout:632 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:632.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:633 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:633.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:634 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:634.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:635 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:635.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:636 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:636.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:637 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:637.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:638 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:638.out 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:639 2024-02-18T11:50:36.986 INFO:teuthology.orchestra.run.smithi116.stdout:639.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:640 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:640.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:641 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:641.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:642 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:642.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:643 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:643.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:644 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:644.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:645 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:645.out 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:646 2024-02-18T11:50:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:646.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:647 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:647.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:648 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:648.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:649 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:649.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:650 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:650.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:651 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:651.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:652 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:652.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:653 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:653.out 2024-02-18T11:50:36.988 INFO:teuthology.orchestra.run.smithi116.stdout:654 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:654.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:655 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:655.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:656 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:656.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:657 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:657.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:658 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:658.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:659 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:659.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:660 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:660.out 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:661 2024-02-18T11:50:36.989 INFO:teuthology.orchestra.run.smithi116.stdout:661.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:662 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:662.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:663 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:663.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:664 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:664.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:665 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:665.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:666 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:666.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:667 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:667.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:668 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:668.out 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:669 2024-02-18T11:50:36.990 INFO:teuthology.orchestra.run.smithi116.stdout:669.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:670 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:670.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:671 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:671.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:672 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:672.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:673 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:673.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:674 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:674.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:675 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:675.out 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:676 2024-02-18T11:50:36.991 INFO:teuthology.orchestra.run.smithi116.stdout:676.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:677 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:677.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:678 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:678.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:679 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:679.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:680 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:680.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:681 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:681.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:682 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:682.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:683 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:683.out 2024-02-18T11:50:36.992 INFO:teuthology.orchestra.run.smithi116.stdout:684 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:684.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:685 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:685.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:686 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:686.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:687 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:687.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:688 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:688.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:689 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:689.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:690 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:690.out 2024-02-18T11:50:36.993 INFO:teuthology.orchestra.run.smithi116.stdout:691 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:691.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:692 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:692.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:693 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:693.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:694 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:694.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:695 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:695.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:696 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:696.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:697 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:697.out 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:698 2024-02-18T11:50:36.994 INFO:teuthology.orchestra.run.smithi116.stdout:698.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:699 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:699.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:700 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:700.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:701 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:701.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:702 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:702.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:703 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:703.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:704 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:704.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:705 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:705.out 2024-02-18T11:50:36.995 INFO:teuthology.orchestra.run.smithi116.stdout:706 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:706.out 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:707 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:707.out 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:708 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:708.out 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:709 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:709.out 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:710 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:710.out 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:711 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:711.out 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:712 2024-02-18T11:50:36.996 INFO:teuthology.orchestra.run.smithi116.stdout:712.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:713 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:713.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:714 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:714.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:715 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:715.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:716 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:716.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:717 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:717.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:718 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:718.out 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:719 2024-02-18T11:50:36.997 INFO:teuthology.orchestra.run.smithi116.stdout:719.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:720 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:720.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:721 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:721.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:722 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:722.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:723 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:723.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:724 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:724.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:725 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:725.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:726 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:726.out 2024-02-18T11:50:36.998 INFO:teuthology.orchestra.run.smithi116.stdout:727 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:727.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:728 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:728.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:729 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:729.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:730 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:730.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:731 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:731.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:732 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:732.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:733 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:733.out 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:734 2024-02-18T11:50:36.999 INFO:teuthology.orchestra.run.smithi116.stdout:734.out 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:735 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:735.out 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:736 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:736.out 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:737 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:737.out 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:738 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:738.out 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:739 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:739.out 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:group.list 2024-02-18T11:50:37.000 INFO:teuthology.orchestra.run.smithi116.stdout:Makefile 2024-02-18T11:50:37.001 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-02-18T11:50:37.180 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:50:37.184 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:50:37.184 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:50:37.185 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:50:37.186 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:40.457 INFO:teuthology.orchestra.run.smithi116.stdout:generic/001 3s 2024-02-18T11:50:40.462 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/001 2024-02-18T11:50:40.462 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:50:40.465 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:41.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:50:41.045 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-02-18T11:50:41.218 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:50:41.222 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:50:41.222 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:50:41.223 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:50:41.223 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:41.700 INFO:teuthology.orchestra.run.smithi116.stdout:generic/002 0s 2024-02-18T11:50:41.705 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/002 2024-02-18T11:50:41.705 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:50:41.707 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:41.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:50:41.758 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-02-18T11:50:41.933 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:50:41.936 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:50:41.937 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:50:41.938 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:50:41.938 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:42.138 INFO:teuthology.orchestra.run.smithi116.stdout:generic/003 [not run] atime not maintained by ceph 2024-02-18T11:50:42.141 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/003 2024-02-18T11:50:42.142 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/003 2024-02-18T11:50:42.142 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:50:42.144 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:42.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:50:42.154 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-02-18T11:50:42.324 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:50:42.328 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:50:42.328 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:50:42.329 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:50:42.330 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:42.526 INFO:teuthology.orchestra.run.smithi116.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-02-18T11:50:42.530 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/004 2024-02-18T11:50:42.530 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/004 2024-02-18T11:50:42.530 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:50:42.532 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:42.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:50:42.554 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-02-18T11:50:42.727 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:50:42.731 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:50:42.732 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:50:42.732 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:50:42.733 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:43.063 INFO:teuthology.orchestra.run.smithi116.stdout:generic/005 1s 2024-02-18T11:50:43.068 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/005 2024-02-18T11:50:43.068 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:50:43.071 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:50:43.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:50:43.129 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-02-18T11:50:43.306 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:50:43.309 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:50:43.310 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:50:43.310 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:50:43.311 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:03.431 INFO:teuthology.orchestra.run.smithi116.stdout:generic/006 20s 2024-02-18T11:51:03.570 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/006 2024-02-18T11:51:03.570 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:51:03.573 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:03.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:51:03.630 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-02-18T11:51:03.931 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:51:03.934 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:51:03.935 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:51:03.936 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:51:03.936 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:04.791 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:51:04.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:51:34.973 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:51:34.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:51:44.894 INFO:teuthology.orchestra.run.smithi116.stdout:generic/007 40s 2024-02-18T11:51:44.900 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/007 2024-02-18T11:51:44.900 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:51:44.902 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:45.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:51:45.060 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-02-18T11:51:45.369 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:51:45.373 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:51:45.373 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:51:45.374 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:51:45.374 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:45.647 INFO:teuthology.orchestra.run.smithi116.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-02-18T11:51:45.650 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/008 2024-02-18T11:51:45.650 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/008 2024-02-18T11:51:45.650 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:51:45.653 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:45.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:51:45.666 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-02-18T11:51:45.834 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:51:45.838 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:51:45.838 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:51:45.839 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:51:45.839 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:46.098 INFO:teuthology.orchestra.run.smithi116.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-02-18T11:51:46.102 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/009 2024-02-18T11:51:46.102 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/009 2024-02-18T11:51:46.102 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:51:46.104 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:46.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:51:46.119 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-02-18T11:51:46.297 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:51:46.300 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:51:46.300 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:51:46.301 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:51:46.301 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:46.845 INFO:teuthology.orchestra.run.smithi116.stdout:generic/010 0s 2024-02-18T11:51:46.983 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/010 2024-02-18T11:51:46.983 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:51:46.986 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:51:47.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:51:47.032 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-02-18T11:51:47.215 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:51:47.219 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:51:47.219 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:51:47.220 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:51:47.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:52:05.262 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:52:05.265 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:52:12.997 INFO:teuthology.orchestra.run.smithi116.stdout:generic/011 25s 2024-02-18T11:52:13.068 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/011 2024-02-18T11:52:13.069 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:52:13.071 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:52:13.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:52:13.258 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-02-18T11:52:13.428 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:52:13.432 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:52:13.432 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:52:13.433 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:52:13.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:52:13.798 INFO:teuthology.orchestra.run.smithi116.stdout:generic/012 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:52:13.802 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/012 2024-02-18T11:52:13.802 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/012 2024-02-18T11:52:13.802 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:52:13.805 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:52:13.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:52:13.847 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-02-18T11:52:14.031 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:52:14.035 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:52:14.035 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:52:14.036 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:52:14.036 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:52:35.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:52:35.341 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:53:05.769 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:53:05.772 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:53:13.946 INFO:teuthology.orchestra.run.smithi116.stdout:generic/013 59s 2024-02-18T11:53:13.952 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/013 2024-02-18T11:53:13.952 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:13.955 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:13.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:13.985 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-02-18T11:53:14.215 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:14.220 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:14.220 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:14.222 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:14.222 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:14.519 INFO:teuthology.orchestra.run.smithi116.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-02-18T11:53:14.524 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/014 2024-02-18T11:53:14.524 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/014 2024-02-18T11:53:14.524 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:14.527 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:14.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:14.547 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-02-18T11:53:14.766 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:14.771 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:14.771 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:14.773 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:14.773 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:15.046 INFO:teuthology.orchestra.run.smithi116.stdout:generic/015 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T11:53:15.051 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/015 2024-02-18T11:53:15.051 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/015 2024-02-18T11:53:15.051 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:15.054 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:15.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:15.070 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-02-18T11:53:15.329 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:15.333 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:15.334 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:15.335 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:15.335 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:15.728 INFO:teuthology.orchestra.run.smithi116.stdout:generic/016 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:53:15.733 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/016 2024-02-18T11:53:15.733 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/016 2024-02-18T11:53:15.733 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:15.736 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:15.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:15.774 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-02-18T11:53:16.042 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:16.046 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:16.047 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:16.048 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:16.048 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:16.327 INFO:teuthology.orchestra.run.smithi116.stdout:generic/017 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T11:53:16.333 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/017 2024-02-18T11:53:16.333 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/017 2024-02-18T11:53:16.333 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:16.336 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:16.353 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:16.353 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-02-18T11:53:16.595 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:16.600 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:16.601 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:16.602 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:16.602 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:16.941 INFO:teuthology.orchestra.run.smithi116.stdout:generic/018 [not run] defragmentation not supported for fstype "ceph" 2024-02-18T11:53:16.950 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/018 2024-02-18T11:53:16.950 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/018 2024-02-18T11:53:16.951 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:16.953 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:16.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:16.968 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-02-18T11:53:17.211 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:17.216 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:17.217 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:17.218 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:17.218 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:17.479 INFO:teuthology.orchestra.run.smithi116.stdout:generic/019 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:53:17.485 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/019 2024-02-18T11:53:17.486 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/019 2024-02-18T11:53:17.486 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:17.488 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:17.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:17.503 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-02-18T11:53:17.742 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:17.748 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:17.748 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:17.750 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:17.750 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:20.899 INFO:teuthology.orchestra.run.smithi116.stdout:generic/020 3s 2024-02-18T11:53:21.048 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/020 2024-02-18T11:53:21.048 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:21.051 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:21.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:21.087 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-02-18T11:53:21.369 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:21.373 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:21.374 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:21.375 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:21.375 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:21.750 INFO:teuthology.orchestra.run.smithi116.stdout:generic/021 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:53:21.754 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/021 2024-02-18T11:53:21.754 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/021 2024-02-18T11:53:21.754 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:21.757 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:21.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:21.798 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-02-18T11:53:22.077 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:22.081 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:22.082 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:22.083 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:22.083 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:22.466 INFO:teuthology.orchestra.run.smithi116.stdout:generic/022 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:53:22.472 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/022 2024-02-18T11:53:22.472 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/022 2024-02-18T11:53:22.472 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:22.475 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:22.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:22.507 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-02-18T11:53:22.789 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:22.795 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:22.796 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:22.796 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:22.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:24.123 INFO:teuthology.orchestra.run.smithi116.stdout:generic/023 2s 2024-02-18T11:53:24.190 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/023 2024-02-18T11:53:24.190 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:24.193 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:24.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:24.255 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-02-18T11:53:24.527 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:24.531 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:24.532 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:24.533 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:24.533 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:24.818 INFO:teuthology.orchestra.run.smithi116.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-02-18T11:53:24.823 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/024 2024-02-18T11:53:24.823 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/024 2024-02-18T11:53:24.823 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:24.827 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:24.856 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:25.060 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-02-18T11:53:25.293 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:25.298 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:25.299 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:25.300 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:25.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:25.616 INFO:teuthology.orchestra.run.smithi116.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-02-18T11:53:25.618 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/025 2024-02-18T11:53:25.618 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/025 2024-02-18T11:53:25.618 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:25.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:25.660 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-02-18T11:53:25.944 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:25.949 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:25.950 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:25.951 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:25.951 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:26.252 INFO:teuthology.orchestra.run.smithi116.stdout:generic/026 [not run] ceph does not define maximum ACL count 2024-02-18T11:53:26.258 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/026 2024-02-18T11:53:26.258 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/026 2024-02-18T11:53:26.258 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:26.261 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:26.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:26.284 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-02-18T11:53:26.540 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:26.543 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:26.543 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:26.543 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:26.543 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:26.795 INFO:teuthology.orchestra.run.smithi116.stdout:generic/027 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T11:53:26.800 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/027 2024-02-18T11:53:26.800 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/027 2024-02-18T11:53:26.800 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:26.805 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:26.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:26.829 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-02-18T11:53:27.065 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:27.070 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:27.070 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:27.072 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:27.072 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:32.369 INFO:teuthology.orchestra.run.smithi116.stdout:generic/028 5s 2024-02-18T11:53:32.505 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/028 2024-02-18T11:53:32.505 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:32.508 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:32.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:32.531 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-02-18T11:53:32.769 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:32.775 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:32.775 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:32.776 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:32.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:36.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:53:36.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:53:36.167 INFO:teuthology.orchestra.run.smithi116.stdout:generic/029 4s 2024-02-18T11:53:36.217 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/029 2024-02-18T11:53:36.217 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:36.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:36.230 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:36.231 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-02-18T11:53:36.506 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:36.510 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:36.510 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:36.511 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:36.512 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:37.944 INFO:teuthology.orchestra.run.smithi116.stdout:generic/030 1s 2024-02-18T11:53:37.952 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/030 2024-02-18T11:53:37.952 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:37.955 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:37.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:37.966 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-02-18T11:53:38.208 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:38.213 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:38.214 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:38.215 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:38.215 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:38.578 INFO:teuthology.orchestra.run.smithi116.stdout:generic/031 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-02-18T11:53:38.583 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/031 2024-02-18T11:53:38.583 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/031 2024-02-18T11:53:38.583 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:38.585 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:38.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:38.609 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-02-18T11:53:38.844 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:38.849 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:38.850 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:38.851 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:38.851 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:39.126 INFO:teuthology.orchestra.run.smithi116.stdout:generic/032 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:53:39.131 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/032 2024-02-18T11:53:39.131 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/032 2024-02-18T11:53:39.131 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:39.134 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:39.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:39.153 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-02-18T11:53:39.396 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:39.401 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:39.401 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:39.402 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:39.403 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:39.759 INFO:teuthology.orchestra.run.smithi116.stdout:generic/033 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-02-18T11:53:39.764 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/033 2024-02-18T11:53:39.764 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/033 2024-02-18T11:53:39.764 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:39.766 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:39.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:39.788 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-02-18T11:53:40.030 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:40.035 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:40.035 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:40.036 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:40.037 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:40.310 INFO:teuthology.orchestra.run.smithi116.stdout:generic/034 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:53:40.317 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/034 2024-02-18T11:53:40.317 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/034 2024-02-18T11:53:40.317 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:40.321 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:40.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:40.338 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-02-18T11:53:40.577 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:40.582 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:40.582 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:40.584 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:40.584 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:40.921 INFO:teuthology.orchestra.run.smithi116.stdout:generic/035 0s 2024-02-18T11:53:40.928 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/035 2024-02-18T11:53:40.928 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:40.932 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:40.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:40.963 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-02-18T11:53:41.240 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:41.245 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:41.246 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:41.247 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:41.247 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:51.612 INFO:teuthology.orchestra.run.smithi116.stdout:generic/036 10s 2024-02-18T11:53:51.615 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/036 2024-02-18T11:53:51.615 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:51.618 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:51.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:51.654 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-02-18T11:53:51.947 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:51.951 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:51.952 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:51.953 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:51.953 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:56.539 INFO:teuthology.orchestra.run.smithi116.stdout:generic/037 4s 2024-02-18T11:53:56.548 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/037 2024-02-18T11:53:56.548 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:56.551 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:56.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:56.562 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-02-18T11:53:56.796 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:56.802 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:56.802 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:56.804 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:56.804 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:57.073 INFO:teuthology.orchestra.run.smithi116.stdout:generic/038 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:53:57.078 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/038 2024-02-18T11:53:57.078 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/038 2024-02-18T11:53:57.078 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:57.080 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:57.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:57.096 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-02-18T11:53:57.322 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:57.327 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:57.328 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:57.329 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:57.329 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:57.616 INFO:teuthology.orchestra.run.smithi116.stdout:generic/039 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:53:57.621 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/039 2024-02-18T11:53:57.621 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/039 2024-02-18T11:53:57.621 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:57.624 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:57.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:57.643 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-02-18T11:53:57.882 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:57.888 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:57.888 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:57.889 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:57.890 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:58.165 INFO:teuthology.orchestra.run.smithi116.stdout:generic/040 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:53:58.170 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/040 2024-02-18T11:53:58.171 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/040 2024-02-18T11:53:58.171 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:58.173 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:58.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:58.192 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-02-18T11:53:58.421 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:58.428 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:58.428 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:58.429 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:58.430 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:58.712 INFO:teuthology.orchestra.run.smithi116.stdout:generic/041 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/041 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/041 2024-02-18T11:53:58.716 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:58.719 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:58.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:58.741 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-02-18T11:53:58.966 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:58.972 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:58.973 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:58.974 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:58.974 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:59.305 INFO:teuthology.orchestra.run.smithi116.stdout:generic/042 [not run] ceph does not support shutdown 2024-02-18T11:53:59.315 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/042 2024-02-18T11:53:59.316 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/042 2024-02-18T11:53:59.316 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:59.318 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:59.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:59.334 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-02-18T11:53:59.567 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:53:59.573 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:53:59.574 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:53:59.575 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:53:59.575 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:59.906 INFO:teuthology.orchestra.run.smithi116.stdout:generic/043 [not run] ceph does not support shutdown 2024-02-18T11:53:59.914 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/043 2024-02-18T11:53:59.914 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/043 2024-02-18T11:53:59.915 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:53:59.917 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:53:59.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:53:59.932 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-02-18T11:54:00.165 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:00.170 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:00.171 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:00.173 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:00.173 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:00.508 INFO:teuthology.orchestra.run.smithi116.stdout:generic/044 [not run] ceph does not support shutdown 2024-02-18T11:54:00.520 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/044 2024-02-18T11:54:00.520 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/044 2024-02-18T11:54:00.520 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:00.523 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:00.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:00.540 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-02-18T11:54:00.775 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:00.780 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:00.781 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:00.782 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:00.782 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:01.127 INFO:teuthology.orchestra.run.smithi116.stdout:generic/045 [not run] ceph does not support shutdown 2024-02-18T11:54:01.137 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/045 2024-02-18T11:54:01.137 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/045 2024-02-18T11:54:01.137 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:01.140 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:01.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:01.155 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-02-18T11:54:01.405 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:01.411 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:01.412 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:01.413 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:01.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:01.767 INFO:teuthology.orchestra.run.smithi116.stdout:generic/046 [not run] ceph does not support shutdown 2024-02-18T11:54:01.775 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/046 2024-02-18T11:54:01.775 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/046 2024-02-18T11:54:01.775 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:01.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:01.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:01.795 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-02-18T11:54:02.031 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:02.036 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:02.040 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:02.040 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:02.040 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:02.387 INFO:teuthology.orchestra.run.smithi116.stdout:generic/047 [not run] ceph does not support shutdown 2024-02-18T11:54:02.398 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/047 2024-02-18T11:54:02.398 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/047 2024-02-18T11:54:02.399 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:02.401 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:02.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:02.415 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-02-18T11:54:02.651 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:02.656 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:02.657 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:02.658 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:02.658 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:03.016 INFO:teuthology.orchestra.run.smithi116.stdout:generic/048 [not run] ceph does not support shutdown 2024-02-18T11:54:03.024 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/048 2024-02-18T11:54:03.024 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/048 2024-02-18T11:54:03.024 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:03.027 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:03.041 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:03.041 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-02-18T11:54:03.276 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:03.281 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:03.282 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:03.283 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:03.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:03.629 INFO:teuthology.orchestra.run.smithi116.stdout:generic/049 [not run] ceph does not support shutdown 2024-02-18T11:54:03.651 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/049 2024-02-18T11:54:03.651 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/049 2024-02-18T11:54:03.651 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:03.654 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:03.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:03.669 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-02-18T11:54:03.911 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:03.916 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:03.916 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:03.918 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:03.918 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:04.259 INFO:teuthology.orchestra.run.smithi116.stdout:generic/050 [not run] ceph does not support shutdown 2024-02-18T11:54:04.269 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/050 2024-02-18T11:54:04.269 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/050 2024-02-18T11:54:04.270 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:04.272 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:04.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:04.296 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-02-18T11:54:04.527 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:04.532 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:04.532 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:04.533 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:04.534 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:04.877 INFO:teuthology.orchestra.run.smithi116.stdout:generic/051 [not run] ceph does not support shutdown 2024-02-18T11:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/051 2024-02-18T11:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/051 2024-02-18T11:54:04.882 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:04.885 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:04.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:04.903 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-02-18T11:54:05.133 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:05.138 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:05.138 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:05.140 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:05.140 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:05.488 INFO:teuthology.orchestra.run.smithi116.stdout:generic/052 [not run] ceph does not support shutdown 2024-02-18T11:54:05.506 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/052 2024-02-18T11:54:05.506 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/052 2024-02-18T11:54:05.506 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:05.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:05.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:05.525 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-02-18T11:54:05.759 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:05.765 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:05.765 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:05.766 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:05.766 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:06.227 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:54:06.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:54:06.572 INFO:teuthology.orchestra.run.smithi116.stdout:generic/053 1s 2024-02-18T11:54:06.651 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/053 2024-02-18T11:54:06.651 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:06.654 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:06.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:06.666 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-02-18T11:54:06.895 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:06.901 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:06.901 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:06.902 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:06.902 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:07.266 INFO:teuthology.orchestra.run.smithi116.stdout:generic/054 [not run] ceph does not support shutdown 2024-02-18T11:54:07.284 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/054 2024-02-18T11:54:07.284 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/054 2024-02-18T11:54:07.284 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:07.287 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:07.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:07.301 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-02-18T11:54:07.530 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:07.535 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:07.535 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:07.536 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:07.536 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:07.884 INFO:teuthology.orchestra.run.smithi116.stdout:generic/055 [not run] ceph does not support shutdown 2024-02-18T11:54:07.893 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/055 2024-02-18T11:54:07.893 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/055 2024-02-18T11:54:07.893 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:07.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:07.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:07.911 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-02-18T11:54:08.145 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:08.151 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:08.151 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:08.153 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:08.153 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:08.427 INFO:teuthology.orchestra.run.smithi116.stdout:generic/056 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:08.433 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/056 2024-02-18T11:54:08.433 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/056 2024-02-18T11:54:08.433 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:08.436 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:08.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:08.455 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-02-18T11:54:08.677 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:08.681 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:08.682 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:08.683 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:08.683 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:08.969 INFO:teuthology.orchestra.run.smithi116.stdout:generic/057 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:08.973 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/057 2024-02-18T11:54:08.974 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/057 2024-02-18T11:54:08.974 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:08.976 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:08.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:08.992 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-02-18T11:54:09.517 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:09.522 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:09.522 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:09.523 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:09.523 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:09.878 INFO:teuthology.orchestra.run.smithi116.stdout:generic/058 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:54:09.883 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/058 2024-02-18T11:54:09.883 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/058 2024-02-18T11:54:09.883 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:09.885 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:09.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:09.917 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-02-18T11:54:10.209 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:10.214 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:10.214 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:10.216 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:10.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:10.482 INFO:teuthology.orchestra.run.smithi116.stdout:generic/059 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:10.487 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/059 2024-02-18T11:54:10.487 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/059 2024-02-18T11:54:10.487 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:10.490 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:10.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:10.504 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-02-18T11:54:10.746 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:10.750 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:10.751 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:10.752 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:10.752 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:11.099 INFO:teuthology.orchestra.run.smithi116.stdout:generic/060 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:54:11.103 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/060 2024-02-18T11:54:11.103 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/060 2024-02-18T11:54:11.103 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:11.105 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:11.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:11.183 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-02-18T11:54:11.465 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:11.469 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:11.470 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:11.471 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:11.471 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:11.829 INFO:teuthology.orchestra.run.smithi116.stdout:generic/061 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:54:11.834 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/061 2024-02-18T11:54:11.834 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/061 2024-02-18T11:54:11.834 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:11.837 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:11.868 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:11.869 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-02-18T11:54:12.165 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:12.170 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:12.171 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:12.172 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:12.172 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:13.748 INFO:teuthology.orchestra.run.smithi116.stdout:generic/062 1s 2024-02-18T11:54:13.765 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/062 2024-02-18T11:54:13.765 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:13.768 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:13.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:13.784 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-02-18T11:54:14.045 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:14.050 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:14.051 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:14.052 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:14.052 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:14.516 INFO:teuthology.orchestra.run.smithi116.stdout:generic/063 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:54:14.521 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/063 2024-02-18T11:54:14.521 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/063 2024-02-18T11:54:14.521 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:14.524 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:14.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:14.565 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-02-18T11:54:14.860 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:14.864 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:14.864 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:14.865 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:14.866 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:15.140 INFO:teuthology.orchestra.run.smithi116.stdout:generic/064 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T11:54:15.143 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/064 2024-02-18T11:54:15.143 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/064 2024-02-18T11:54:15.143 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:15.146 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:15.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:15.162 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-02-18T11:54:15.393 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:15.398 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:15.398 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:15.399 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:15.399 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:15.685 INFO:teuthology.orchestra.run.smithi116.stdout:generic/065 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:15.690 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/065 2024-02-18T11:54:15.690 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/065 2024-02-18T11:54:15.690 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:15.693 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:15.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:15.711 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-02-18T11:54:15.953 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:15.957 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:15.958 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:15.959 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:15.960 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:16.238 INFO:teuthology.orchestra.run.smithi116.stdout:generic/066 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:16.242 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/066 2024-02-18T11:54:16.242 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/066 2024-02-18T11:54:16.242 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:16.244 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:16.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:16.263 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-02-18T11:54:16.492 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:16.496 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:16.497 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:16.498 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:16.498 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:16.784 INFO:teuthology.orchestra.run.smithi116.stdout:generic/067 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:16.789 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/067 2024-02-18T11:54:16.789 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/067 2024-02-18T11:54:16.789 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:16.793 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:16.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:16.811 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-02-18T11:54:17.051 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:17.056 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:17.056 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:17.058 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:17.058 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:17.333 INFO:teuthology.orchestra.run.smithi116.stdout:generic/068 [not run] ceph does not support freezing 2024-02-18T11:54:17.338 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/068 2024-02-18T11:54:17.338 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/068 2024-02-18T11:54:17.338 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:17.341 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:17.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:17.354 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-02-18T11:54:17.585 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:17.590 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:17.591 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:17.592 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:17.592 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:24.372 INFO:teuthology.orchestra.run.smithi116.stdout:generic/069 7s 2024-02-18T11:54:24.380 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/069 2024-02-18T11:54:24.381 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:24.382 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:24.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:24.395 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-02-18T11:54:24.654 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:24.659 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:24.659 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:24.661 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:24.661 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:36.590 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:54:36.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:54:49.135 INFO:teuthology.orchestra.run.smithi116.stdout:generic/070 25s 2024-02-18T11:54:49.144 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/070 2024-02-18T11:54:49.144 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:49.146 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:49.244 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:49.244 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-02-18T11:54:49.515 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:49.520 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:49.520 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:49.522 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:49.522 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:49.790 INFO:teuthology.orchestra.run.smithi116.stdout:generic/071 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T11:54:49.795 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/071 2024-02-18T11:54:49.796 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/071 2024-02-18T11:54:49.796 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:49.798 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:49.820 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:49.820 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-02-18T11:54:50.076 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:50.080 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:50.081 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:50.082 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:50.082 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:50.429 INFO:teuthology.orchestra.run.smithi116.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-02-18T11:54:50.434 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/072 2024-02-18T11:54:50.434 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/072 2024-02-18T11:54:50.434 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:50.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:50.455 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:50.455 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-02-18T11:54:50.678 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:50.683 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:50.684 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:50.685 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:50.685 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:50.958 INFO:teuthology.orchestra.run.smithi116.stdout:generic/073 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:54:50.962 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/073 2024-02-18T11:54:50.963 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/073 2024-02-18T11:54:50.963 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:54:50.965 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:54:50.980 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:54:50.980 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-02-18T11:54:51.212 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:54:51.217 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:54:51.218 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:54:51.219 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:54:51.219 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:06.774 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:55:06.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:55:16.994 INFO:teuthology.orchestra.run.smithi116.stdout:generic/074 25s 2024-02-18T11:55:17.007 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/074 2024-02-18T11:55:17.007 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:17.011 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:18.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:18.213 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-02-18T11:55:18.514 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:18.518 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:18.518 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:18.520 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:18.520 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:36.990 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:55:36.993 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:55:46.662 INFO:teuthology.orchestra.run.smithi116.stdout:generic/075 28s 2024-02-18T11:55:46.791 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/075 2024-02-18T11:55:46.791 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:46.794 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:46.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:46.830 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-02-18T11:55:47.169 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:47.175 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:47.176 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:47.177 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:47.177 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:47.443 INFO:teuthology.orchestra.run.smithi116.stdout:generic/076 [not run] require 172.21.15.116:6789:/scratch to be local device 2024-02-18T11:55:47.448 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/076 2024-02-18T11:55:47.448 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/076 2024-02-18T11:55:47.448 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:47.451 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:47.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:47.465 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-02-18T11:55:47.701 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:47.707 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:47.707 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:47.709 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:47.709 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:50.480 INFO:teuthology.orchestra.run.smithi116.stdout:generic/077 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T11:55:50.485 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/077 2024-02-18T11:55:50.485 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/077 2024-02-18T11:55:50.485 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:50.488 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:50.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:50.524 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-02-18T11:55:50.851 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:50.856 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:50.856 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:50.857 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:50.857 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:51.135 INFO:teuthology.orchestra.run.smithi116.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-02-18T11:55:51.140 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/078 2024-02-18T11:55:51.140 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/078 2024-02-18T11:55:51.140 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:51.143 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:51.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:51.175 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-02-18T11:55:51.497 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:51.502 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:51.502 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:51.503 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:51.503 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:51.768 INFO:teuthology.orchestra.run.smithi116.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-02-18T11:55:51.773 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/079 2024-02-18T11:55:51.773 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/079 2024-02-18T11:55:51.773 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:51.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:51.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:51.798 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-02-18T11:55:52.021 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:52.025 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:52.025 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:52.027 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:52.027 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:54.333 INFO:teuthology.orchestra.run.smithi116.stdout:generic/080 2s 2024-02-18T11:55:54.342 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/080 2024-02-18T11:55:54.343 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:54.346 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:54.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:54.377 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-02-18T11:55:54.610 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:54.615 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:54.616 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:54.617 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:54.617 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:54.921 INFO:teuthology.orchestra.run.smithi116.stdout:generic/081 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:55:54.925 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/081 2024-02-18T11:55:54.925 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/081 2024-02-18T11:55:54.926 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:54.928 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:54.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:54.944 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-02-18T11:55:55.169 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:55.174 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:55.175 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:55.176 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:55.176 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:55.445 INFO:teuthology.orchestra.run.smithi116.stdout:generic/082 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T11:55:55.450 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/082 2024-02-18T11:55:55.451 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/082 2024-02-18T11:55:55.451 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:55.454 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:55.471 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:55.472 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-02-18T11:55:55.706 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:55.711 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:55.712 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:55.713 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:55.713 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:55.979 INFO:teuthology.orchestra.run.smithi116.stdout:generic/083 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T11:55:55.983 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/083 2024-02-18T11:55:55.983 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/083 2024-02-18T11:55:55.983 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:55:55.986 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:55:56.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:55:56.002 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-02-18T11:55:56.225 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:55:56.229 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:55:56.229 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:55:56.230 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:55:56.231 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:01.839 INFO:teuthology.orchestra.run.smithi116.stdout:generic/084 5s 2024-02-18T11:56:01.852 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/084 2024-02-18T11:56:01.853 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:56:01.856 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:01.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:56:01.869 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-02-18T11:56:02.122 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:56:02.126 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:56:02.127 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:56:02.128 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:56:02.128 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:02.391 INFO:teuthology.orchestra.run.smithi116.stdout:generic/085 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T11:56:02.396 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/085 2024-02-18T11:56:02.396 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/085 2024-02-18T11:56:02.396 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:56:02.398 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:02.422 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:56:02.422 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-02-18T11:56:02.674 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:56:02.679 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:56:02.680 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:56:02.681 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:56:02.681 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:02.947 INFO:teuthology.orchestra.run.smithi116.stdout:generic/086 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T11:56:02.952 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/086 2024-02-18T11:56:02.952 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/086 2024-02-18T11:56:02.952 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:56:02.955 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:02.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:56:02.973 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-02-18T11:56:03.206 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:56:03.210 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:56:03.211 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:56:03.212 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:56:03.212 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:03.547 INFO:teuthology.orchestra.run.smithi116.stdout:generic/087 0s 2024-02-18T11:56:03.554 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/087 2024-02-18T11:56:03.554 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:56:03.557 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:03.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:56:03.607 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-02-18T11:56:03.931 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:56:03.936 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:56:03.936 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:56:03.937 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:56:03.937 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:04.234 INFO:teuthology.orchestra.run.smithi116.stdout:generic/088 0s 2024-02-18T11:56:04.241 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/088 2024-02-18T11:56:04.241 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T11:56:04.244 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:04.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T11:56:04.282 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-02-18T11:56:04.597 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T11:56:04.605 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T11:56:04.605 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T11:56:04.607 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T11:56:04.607 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T11:56:07.113 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:56:07.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:56:37.487 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:56:37.490 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:56:37.543 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T11:56:37.549+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173098) UID: 0 2024-02-18T11:56:37.543 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T11:56:37.549+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173098) UID: 0 2024-02-18T11:59:51.652 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T11:59:51.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:00:21.980 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:00:21.985 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:00:52.486 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:00:52.490 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:00:52.545 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T12:00:52.547+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173379) UID: 0 2024-02-18T12:00:52.584 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T12:00:52.547+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 173379) UID: 0 2024-02-18T12:03:48.388 INFO:teuthology.orchestra.run.smithi116.stdout:generic/089 464s 2024-02-18T12:03:48.447 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/089 2024-02-18T12:03:48.447 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:03:48.450 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:03:49.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:03:49.311 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-02-18T12:03:50.137 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:03:50.146 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:03:50.146 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:03:50.151 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:03:50.152 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:03:50.609 INFO:teuthology.orchestra.run.smithi116.stdout:generic/090 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:03:50.613 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/090 2024-02-18T12:03:50.613 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/090 2024-02-18T12:03:50.613 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:03:50.615 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:03:50.630 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:03:50.630 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-02-18T12:03:50.810 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:03:50.814 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:03:50.814 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:03:50.815 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:03:50.815 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:04:01.417 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:04:01.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:04:31.688 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:04:31.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:05:01.758 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:05:01.762 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:05:17.090 INFO:teuthology.orchestra.run.smithi116.stdout:generic/091 87s 2024-02-18T12:05:17.095 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/091 2024-02-18T12:05:17.095 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:17.097 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:17.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:17.143 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-02-18T12:05:17.521 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:17.524 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:17.525 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:17.526 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:17.526 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:17.747 INFO:teuthology.orchestra.run.smithi116.stdout:generic/092 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:05:17.751 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/092 2024-02-18T12:05:17.751 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/092 2024-02-18T12:05:17.751 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:17.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:17.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:17.767 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-02-18T12:05:17.941 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:17.944 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:17.945 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:17.946 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:17.946 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:18.752 INFO:teuthology.orchestra.run.smithi116.stdout:generic/093 0s 2024-02-18T12:05:18.758 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/093 2024-02-18T12:05:18.758 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:18.760 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:18.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:18.809 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-02-18T12:05:19.183 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:19.187 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:19.187 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:19.188 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:19.188 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:19.439 INFO:teuthology.orchestra.run.smithi116.stdout:generic/094 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T12:05:19.443 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/094 2024-02-18T12:05:19.443 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/094 2024-02-18T12:05:19.443 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:19.445 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:19.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:19.482 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-02-18T12:05:19.845 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:19.848 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:19.848 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:19.850 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:19.850 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:31.482 INFO:teuthology.orchestra.run.smithi116.stdout:generic/095 12s 2024-02-18T12:05:31.491 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/095 2024-02-18T12:05:31.491 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:31.493 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:31.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:31.506 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-02-18T12:05:31.725 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:31.729 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:31.729 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:31.730 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:31.730 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:31.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:05:31.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:05:31.999 INFO:teuthology.orchestra.run.smithi116.stdout:generic/096 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-02-18T12:05:32.003 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/096 2024-02-18T12:05:32.003 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/096 2024-02-18T12:05:32.003 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:32.005 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:32.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:32.021 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-02-18T12:05:32.242 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:32.244 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:32.246 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:32.246 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:32.246 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:32.883 INFO:teuthology.orchestra.run.smithi116.stdout:generic/097 0s 2024-02-18T12:05:32.888 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/097 2024-02-18T12:05:32.888 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:32.891 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:32.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:32.905 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-02-18T12:05:33.077 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:33.081 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:33.081 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:33.082 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:33.082 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:37.544 INFO:teuthology.orchestra.run.smithi116.stdout:generic/098 4s 2024-02-18T12:05:37.550 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/098 2024-02-18T12:05:37.550 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:37.552 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:37.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:37.561 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-02-18T12:05:37.736 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:37.742 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:37.742 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:37.743 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:37.743 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:38.517 INFO:teuthology.orchestra.run.smithi116.stdout:generic/099 1s 2024-02-18T12:05:38.522 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/099 2024-02-18T12:05:38.523 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:38.525 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:38.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:38.567 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-02-18T12:05:38.979 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:38.983 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:38.983 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:38.984 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:38.984 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:49.277 INFO:teuthology.orchestra.run.smithi116.stdout:generic/100 10s 2024-02-18T12:05:49.281 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/100 2024-02-18T12:05:49.281 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:49.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:52.126 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:52.126 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-02-18T12:05:52.441 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:52.445 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:52.445 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:52.446 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:52.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:52.661 INFO:teuthology.orchestra.run.smithi116.stdout:generic/101 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:05:52.664 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/101 2024-02-18T12:05:52.664 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/101 2024-02-18T12:05:52.664 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:52.667 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:52.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:52.680 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-02-18T12:05:52.859 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:52.863 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:52.863 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:52.864 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:52.864 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:53.068 INFO:teuthology.orchestra.run.smithi116.stdout:generic/102 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:05:53.072 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/102 2024-02-18T12:05:53.072 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/102 2024-02-18T12:05:53.073 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:53.074 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:53.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:53.087 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-02-18T12:05:53.271 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:53.274 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:53.276 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:53.276 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:53.276 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:53.503 INFO:teuthology.orchestra.run.smithi116.stdout:generic/103 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:05:53.507 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/103 2024-02-18T12:05:53.507 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/103 2024-02-18T12:05:53.508 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:53.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:53.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:53.558 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-02-18T12:05:53.964 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:53.968 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:53.969 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:53.969 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:53.970 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:54.190 INFO:teuthology.orchestra.run.smithi116.stdout:generic/104 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:05:54.193 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/104 2024-02-18T12:05:54.193 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/104 2024-02-18T12:05:54.194 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:54.196 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:54.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:54.211 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-02-18T12:05:54.387 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:54.391 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:54.391 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:54.392 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:54.392 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:54.823 INFO:teuthology.orchestra.run.smithi116.stdout:generic/105 0s 2024-02-18T12:05:54.830 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/105 2024-02-18T12:05:54.830 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:54.833 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:54.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:54.845 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-02-18T12:05:55.159 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:55.163 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:55.163 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:55.164 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:55.164 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:55.388 INFO:teuthology.orchestra.run.smithi116.stdout:generic/106 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:05:55.392 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/106 2024-02-18T12:05:55.392 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/106 2024-02-18T12:05:55.392 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:55.394 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:55.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:55.410 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-02-18T12:05:55.709 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:55.713 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:55.713 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:55.714 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:55.715 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:generic/107 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:05:55.938 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/107 2024-02-18T12:05:55.938 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/107 2024-02-18T12:05:55.938 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:55.941 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:55.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:55.956 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-02-18T12:05:56.137 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:56.141 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:56.141 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:56.143 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:56.143 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:56.566 INFO:teuthology.orchestra.run.smithi116.stdout:generic/108 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:05:56.570 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/108 2024-02-18T12:05:56.570 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/108 2024-02-18T12:05:56.571 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:05:56.573 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:05:56.585 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:05:56.585 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-02-18T12:05:56.759 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:05:56.763 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:05:56.763 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:05:56.765 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:05:56.765 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:02.003 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:06:02.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:06:02.038 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T12:06:02.039+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 199396) UID: 0 2024-02-18T12:06:02.039 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T12:06:02.039+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 199396) UID: 0 2024-02-18T12:06:04.531 INFO:teuthology.orchestra.run.smithi116.stdout:generic/109 8s 2024-02-18T12:06:04.546 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/109 2024-02-18T12:06:04.546 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:06:04.548 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:04.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:06:04.568 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-02-18T12:06:05.384 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:06:05.387 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:06:05.389 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:06:05.389 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:06:05.390 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:22.795 INFO:teuthology.orchestra.run.smithi116.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:06:22.799 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/110 2024-02-18T12:06:22.799 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/110 2024-02-18T12:06:22.799 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:06:22.825 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:22.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:06:22.968 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-02-18T12:06:23.429 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:06:23.435 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:06:23.435 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:06:23.437 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:06:23.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:23.811 INFO:teuthology.orchestra.run.smithi116.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:06:23.815 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/111 2024-02-18T12:06:23.815 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/111 2024-02-18T12:06:23.816 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:06:23.818 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:23.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:06:23.867 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-02-18T12:06:24.267 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:06:24.273 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:06:24.273 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:06:24.274 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:06:24.274 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:44.583 INFO:teuthology.orchestra.run.smithi116.stdout:generic/112 20s 2024-02-18T12:06:44.651 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/112 2024-02-18T12:06:44.651 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:06:44.651 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:06:44.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:06:44.693 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-02-18T12:06:45.103 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:06:45.107 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:06:45.107 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:06:45.108 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:06:45.109 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:00.697 INFO:teuthology.orchestra.run.smithi116.stdout:generic/113 15s 2024-02-18T12:07:00.701 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/113 2024-02-18T12:07:00.702 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:00.704 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:00.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:00.747 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-02-18T12:07:01.179 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:01.186 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:01.186 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:01.189 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:01.189 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:01.491 INFO:teuthology.orchestra.run.smithi116.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-02-18T12:07:01.495 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/114 2024-02-18T12:07:01.496 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/114 2024-02-18T12:07:01.496 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:01.498 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:01.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:01.520 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-02-18T12:07:01.704 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:01.708 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:01.708 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:01.710 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:01.710 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:01.938 INFO:teuthology.orchestra.run.smithi116.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:07:01.942 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/115 2024-02-18T12:07:01.942 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/115 2024-02-18T12:07:01.942 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:01.945 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:01.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:01.997 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-02-18T12:07:02.408 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:02.411 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:02.412 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:02.413 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:02.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:02.647 INFO:teuthology.orchestra.run.smithi116.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:07:02.651 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/116 2024-02-18T12:07:02.651 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/116 2024-02-18T12:07:02.651 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:02.653 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:02.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:02.709 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-02-18T12:07:03.110 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:03.114 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:03.115 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:03.116 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:03.116 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:19.192 INFO:teuthology.orchestra.run.smithi116.stdout:generic/117 16s 2024-02-18T12:07:19.201 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/117 2024-02-18T12:07:19.202 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:19.205 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:19.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:19.218 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-02-18T12:07:19.421 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:19.425 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:19.425 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:19.426 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:19.426 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:19.988 INFO:teuthology.orchestra.run.smithi116.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:07:19.992 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/118 2024-02-18T12:07:19.992 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/118 2024-02-18T12:07:19.992 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:19.994 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:20.038 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:20.038 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-02-18T12:07:20.440 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:20.444 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:20.445 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:20.446 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:20.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:20.676 INFO:teuthology.orchestra.run.smithi116.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:07:20.680 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/119 2024-02-18T12:07:20.680 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/119 2024-02-18T12:07:20.680 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:20.682 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:20.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:20.728 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-02-18T12:07:21.134 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:21.137 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:21.138 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:21.139 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:21.139 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:21.352 INFO:teuthology.orchestra.run.smithi116.stdout:generic/120 [not run] atime not maintained by ceph 2024-02-18T12:07:21.356 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/120 2024-02-18T12:07:21.356 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/120 2024-02-18T12:07:21.356 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:21.358 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:21.368 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:21.369 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-02-18T12:07:21.540 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:21.543 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:21.544 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:21.545 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:21.545 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:21.777 INFO:teuthology.orchestra.run.smithi116.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:07:21.781 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/121 2024-02-18T12:07:21.781 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/121 2024-02-18T12:07:21.781 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:21.845 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:21.845 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-02-18T12:07:22.225 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:22.229 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:22.229 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:22.230 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:22.230 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:22.456 INFO:teuthology.orchestra.run.smithi116.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:07:22.460 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/122 2024-02-18T12:07:22.460 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/122 2024-02-18T12:07:22.460 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:22.462 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:22.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:22.514 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-02-18T12:07:22.900 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:22.904 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:22.904 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:22.905 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:22.905 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:23.400 INFO:teuthology.orchestra.run.smithi116.stdout:generic/123 1s 2024-02-18T12:07:23.405 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/123 2024-02-18T12:07:23.405 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:23.407 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:23.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:23.465 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-02-18T12:07:23.856 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:23.860 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:23.861 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:23.861 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:25.622 INFO:teuthology.orchestra.run.smithi116.stdout:generic/124 2s 2024-02-18T12:07:25.634 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/124 2024-02-18T12:07:25.634 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:07:25.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:07:26.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:07:26.236 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-02-18T12:07:26.607 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:07:26.611 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:07:26.612 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:07:26.613 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:07:26.613 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:08:28.305 INFO:teuthology.orchestra.run.smithi116.stdout:generic/125 62s 2024-02-18T12:08:28.324 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/125 2024-02-18T12:08:28.324 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:08:28.327 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:08:28.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:08:28.369 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-02-18T12:08:28.750 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:08:28.753 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:08:28.754 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:08:28.755 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:08:28.755 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:08:29.119 INFO:teuthology.orchestra.run.smithi116.stdout:generic/126 1s 2024-02-18T12:08:29.124 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/126 2024-02-18T12:08:29.124 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:08:29.126 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:08:29.193 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:08:29.193 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-02-18T12:08:29.546 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:08:29.550 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:08:29.550 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:08:29.551 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:08:29.551 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:08:42.860 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:08:42.863 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:09:12.931 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:09:12.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:09:42.988 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:09:42.993 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:10:13.227 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:10:13.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:10:43.375 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:10:43.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:11:13.745 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:11:13.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:11:43.803 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:11:43.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:12:13.909 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:12:13.912 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:12:43.991 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:12:43.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:13:14.074 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:13:14.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:13:44.179 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:13:44.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:14:14.318 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:14:14.321 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:14:44.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:14:44.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:15:14.546 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:15:14.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:15:44.851 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:15:44.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:16:14.923 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:16:14.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:16:45.198 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:16:45.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:17:15.309 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:17:15.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:17:45.606 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:17:45.609 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:18:15.673 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:18:15.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:18:45.816 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:18:45.819 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:18:53.144 INFO:teuthology.orchestra.run.smithi116.stdout:generic/127 624s 2024-02-18T12:18:53.149 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/127 2024-02-18T12:18:53.149 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:18:53.151 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:53.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:18:53.198 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-02-18T12:18:53.554 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:18:53.558 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:18:53.558 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:18:53.559 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:18:53.559 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:54.407 INFO:teuthology.orchestra.run.smithi116.stdout:generic/128 1s 2024-02-18T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/128 2024-02-18T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:18:54.420 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:54.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:18:54.432 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-02-18T12:18:54.673 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:18:54.677 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:18:54.677 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:18:54.678 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:18:54.678 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:54.888 INFO:teuthology.orchestra.run.smithi116.stdout:generic/129 [not run] Sparse files are not supported or do not work as expected 2024-02-18T12:18:54.892 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/129 2024-02-18T12:18:54.892 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/129 2024-02-18T12:18:54.892 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:18:54.894 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:54.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:18:54.913 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-02-18T12:18:55.088 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:18:55.092 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:18:55.092 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:18:55.094 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:18:55.094 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:55.312 INFO:teuthology.orchestra.run.smithi116.stdout:generic/130 [not run] Sparse files are not supported or do not work as expected 2024-02-18T12:18:55.316 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/130 2024-02-18T12:18:55.316 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/130 2024-02-18T12:18:55.316 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:18:55.318 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:55.339 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:18:55.339 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-02-18T12:18:55.515 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:18:55.518 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:18:55.519 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:18:55.520 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:18:55.520 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:56.840 INFO:teuthology.orchestra.run.smithi116.stdout:generic/131 1s 2024-02-18T12:18:56.846 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/131 2024-02-18T12:18:56.846 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:18:56.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:18:56.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:18:56.903 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-02-18T12:18:57.257 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:18:57.260 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:18:57.261 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:18:57.262 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:18:57.262 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:19:08.967 INFO:teuthology.orchestra.run.smithi116.stdout:generic/132 11s 2024-02-18T12:19:08.972 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/132 2024-02-18T12:19:08.972 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:19:08.974 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:19:08.983 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:19:08.983 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-02-18T12:19:09.163 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:19:09.167 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:19:09.167 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:19:09.168 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:19:09.168 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:19:15.921 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:19:15.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:19:45.994 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:19:45.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:20:16.090 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:20:16.094 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:20:46.239 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:20:46.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:20:46.274 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T12:20:46.273+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212855) UID: 0 2024-02-18T12:20:46.275 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T12:20:46.273+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212855) UID: 0 2024-02-18T12:20:46.275 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T12:20:46.273+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212855) UID: 0 2024-02-18T12:20:46.275 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T12:20:46.273+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212855) UID: 0 2024-02-18T12:22:46.744 INFO:teuthology.orchestra.run.smithi116.stdout:generic/133 217s 2024-02-18T12:22:46.752 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/133 2024-02-18T12:22:46.752 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:46.755 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:46.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:46.833 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-02-18T12:22:47.366 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:47.371 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:47.372 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:47.375 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:47.375 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:47.828 INFO:teuthology.orchestra.run.smithi116.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:47.831 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/134 2024-02-18T12:22:47.832 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/134 2024-02-18T12:22:47.832 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:47.834 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:47.885 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:47.885 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-02-18T12:22:48.299 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:48.303 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:48.304 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:48.305 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:48.305 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:49.061 INFO:teuthology.orchestra.run.smithi116.stdout:generic/135 1s 2024-02-18T12:22:49.068 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/135 2024-02-18T12:22:49.068 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:49.071 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:49.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:49.079 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-02-18T12:22:49.255 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:49.259 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:49.259 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:49.261 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:49.261 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:49.483 INFO:teuthology.orchestra.run.smithi116.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:22:49.487 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/136 2024-02-18T12:22:49.488 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/136 2024-02-18T12:22:49.488 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:49.490 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:49.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:49.543 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-02-18T12:22:49.926 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:49.931 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:49.932 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:49.934 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:49.934 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:50.172 INFO:teuthology.orchestra.run.smithi116.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:50.174 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/137 2024-02-18T12:22:50.174 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/137 2024-02-18T12:22:50.174 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:50.176 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:50.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:50.228 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-02-18T12:22:50.615 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:50.621 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:50.621 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:50.623 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:50.623 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:50.854 INFO:teuthology.orchestra.run.smithi116.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:50.858 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/138 2024-02-18T12:22:50.858 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/138 2024-02-18T12:22:50.858 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:50.860 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:50.911 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:50.911 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-02-18T12:22:51.282 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:51.292 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:51.293 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:51.295 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:51.295 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:51.529 INFO:teuthology.orchestra.run.smithi116.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:51.533 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/139 2024-02-18T12:22:51.533 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/139 2024-02-18T12:22:51.533 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:51.536 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:51.587 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:51.587 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-02-18T12:22:51.987 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:51.991 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:51.991 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:51.992 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:51.992 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:52.220 INFO:teuthology.orchestra.run.smithi116.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:52.221 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/140 2024-02-18T12:22:52.221 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/140 2024-02-18T12:22:52.221 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:52.224 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:52.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:52.285 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-02-18T12:22:52.650 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:52.656 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:52.656 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:52.657 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:52.657 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:52.990 INFO:teuthology.orchestra.run.smithi116.stdout:generic/141 0s 2024-02-18T12:22:52.996 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/141 2024-02-18T12:22:52.996 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:52.998 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:53.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:53.006 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-02-18T12:22:53.182 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:53.186 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:53.186 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:53.187 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:53.187 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:53.476 INFO:teuthology.orchestra.run.smithi116.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:53.480 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/142 2024-02-18T12:22:53.480 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/142 2024-02-18T12:22:53.480 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:53.482 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:53.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:53.535 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-02-18T12:22:53.926 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:53.930 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:53.930 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:53.932 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:53.932 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:54.331 INFO:teuthology.orchestra.run.smithi116.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:54.335 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/143 2024-02-18T12:22:54.335 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/143 2024-02-18T12:22:54.335 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:54.338 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:54.380 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:54.380 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-02-18T12:22:54.753 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:54.771 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:54.772 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:54.777 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:54.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:55.007 INFO:teuthology.orchestra.run.smithi116.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/144 2024-02-18T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/144 2024-02-18T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:55.013 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:55.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:55.073 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-02-18T12:22:55.458 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:55.462 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:55.462 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:55.463 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:55.463 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:55.719 INFO:teuthology.orchestra.run.smithi116.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:55.723 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/145 2024-02-18T12:22:55.723 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/145 2024-02-18T12:22:55.723 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:55.725 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:55.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:55.773 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-02-18T12:22:56.171 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:56.176 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:56.176 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:56.177 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:56.177 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:56.407 INFO:teuthology.orchestra.run.smithi116.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:56.411 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/146 2024-02-18T12:22:56.411 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/146 2024-02-18T12:22:56.411 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:56.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:56.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:56.459 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-02-18T12:22:56.868 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:56.871 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:56.872 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:56.873 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:56.873 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:57.096 INFO:teuthology.orchestra.run.smithi116.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:57.099 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/147 2024-02-18T12:22:57.099 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/147 2024-02-18T12:22:57.100 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:57.102 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:57.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:57.148 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-02-18T12:22:57.539 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:57.543 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:57.544 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:57.545 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:57.545 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:57.779 INFO:teuthology.orchestra.run.smithi116.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:57.783 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/148 2024-02-18T12:22:57.783 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/148 2024-02-18T12:22:57.783 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:57.785 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:57.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:57.838 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-02-18T12:22:58.239 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:58.243 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:58.243 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:58.244 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:58.244 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:58.482 INFO:teuthology.orchestra.run.smithi116.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:58.485 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/149 2024-02-18T12:22:58.486 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/149 2024-02-18T12:22:58.486 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:58.488 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:58.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:58.537 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-02-18T12:22:58.914 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:58.919 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:58.919 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:58.920 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:58.920 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:59.147 INFO:teuthology.orchestra.run.smithi116.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/150 2024-02-18T12:22:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/150 2024-02-18T12:22:59.151 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:59.154 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:59.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:59.201 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-02-18T12:22:59.610 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:22:59.614 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:22:59.614 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:22:59.615 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:22:59.615 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:59.883 INFO:teuthology.orchestra.run.smithi116.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:22:59.887 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/151 2024-02-18T12:22:59.887 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/151 2024-02-18T12:22:59.887 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:22:59.889 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:22:59.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:22:59.942 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-02-18T12:23:00.303 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:00.310 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:00.310 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:00.312 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:00.312 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:00.554 INFO:teuthology.orchestra.run.smithi116.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:00.558 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/152 2024-02-18T12:23:00.558 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/152 2024-02-18T12:23:00.558 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:00.560 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:00.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:00.622 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-02-18T12:23:01.018 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:01.022 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:01.022 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:01.023 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:01.023 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:01.250 INFO:teuthology.orchestra.run.smithi116.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:01.254 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/153 2024-02-18T12:23:01.254 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/153 2024-02-18T12:23:01.254 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:01.257 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:01.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:01.299 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-02-18T12:23:01.698 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:01.702 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:01.702 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:01.703 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:01.703 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:01.933 INFO:teuthology.orchestra.run.smithi116.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:01.936 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/154 2024-02-18T12:23:01.936 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/154 2024-02-18T12:23:01.937 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:01.939 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:01.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:01.991 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-02-18T12:23:02.356 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:02.360 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:02.361 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:02.362 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:02.362 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:02.613 INFO:teuthology.orchestra.run.smithi116.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/155 2024-02-18T12:23:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/155 2024-02-18T12:23:02.617 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:02.619 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:02.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:02.665 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-02-18T12:23:03.071 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:03.075 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:03.075 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:03.076 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:03.076 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:03.330 INFO:teuthology.orchestra.run.smithi116.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:03.334 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/156 2024-02-18T12:23:03.334 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/156 2024-02-18T12:23:03.334 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:03.336 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:03.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:03.389 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-02-18T12:23:03.766 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:03.770 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:03.770 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:03.771 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:03.771 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:04.004 INFO:teuthology.orchestra.run.smithi116.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:04.008 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/157 2024-02-18T12:23:04.008 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/157 2024-02-18T12:23:04.008 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:04.010 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:04.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:04.071 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-02-18T12:23:04.468 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:04.472 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:04.472 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:04.473 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:04.473 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:04.700 INFO:teuthology.orchestra.run.smithi116.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:23:04.704 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/158 2024-02-18T12:23:04.704 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/158 2024-02-18T12:23:04.704 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:04.706 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:04.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:04.758 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-02-18T12:23:05.151 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:05.155 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:05.155 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:05.156 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:05.156 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:05.406 INFO:teuthology.orchestra.run.smithi116.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-02-18T12:23:05.409 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/159 2024-02-18T12:23:05.410 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/159 2024-02-18T12:23:05.410 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:05.412 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:05.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:05.428 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-02-18T12:23:05.610 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:05.614 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:05.615 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:05.615 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:05.616 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:05.855 INFO:teuthology.orchestra.run.smithi116.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-02-18T12:23:05.859 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/160 2024-02-18T12:23:05.859 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/160 2024-02-18T12:23:05.859 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:05.861 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:05.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:05.879 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-02-18T12:23:06.062 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:06.066 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:06.066 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:06.067 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:06.067 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:06.404 INFO:teuthology.orchestra.run.smithi116.stdout:generic/161 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:06.410 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/161 2024-02-18T12:23:06.410 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/161 2024-02-18T12:23:06.410 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:06.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:06.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:06.486 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-02-18T12:23:06.821 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:06.824 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:06.825 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:06.826 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:06.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:07.116 INFO:teuthology.orchestra.run.smithi116.stdout:generic/162 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-02-18T12:23:07.162 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/162 2024-02-18T12:23:07.162 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/162 2024-02-18T12:23:07.162 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:07.164 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:07.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:07.286 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-02-18T12:23:07.590 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:07.593 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:07.594 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:07.594 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:07.595 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:07.889 INFO:teuthology.orchestra.run.smithi116.stdout:generic/163 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-02-18T12:23:07.935 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/163 2024-02-18T12:23:07.935 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/163 2024-02-18T12:23:07.935 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:07.939 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:08.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:08.069 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-02-18T12:23:08.365 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:08.368 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:08.369 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:08.370 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:08.370 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:08.707 INFO:teuthology.orchestra.run.smithi116.stdout:generic/164 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:08.712 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/164 2024-02-18T12:23:08.712 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/164 2024-02-18T12:23:08.712 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:08.721 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:08.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:08.758 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-02-18T12:23:09.104 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:09.108 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:09.108 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:09.109 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:09.109 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:09.516 INFO:teuthology.orchestra.run.smithi116.stdout:generic/165 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:09.523 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/165 2024-02-18T12:23:09.523 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/165 2024-02-18T12:23:09.523 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:09.525 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:09.608 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:09.609 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-02-18T12:23:09.925 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:09.929 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:09.929 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:09.930 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:09.930 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:10.288 INFO:teuthology.orchestra.run.smithi116.stdout:generic/166 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:10.294 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/166 2024-02-18T12:23:10.294 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/166 2024-02-18T12:23:10.294 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:10.297 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:10.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:10.362 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-02-18T12:23:10.687 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:10.691 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:10.692 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:10.692 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:10.692 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:11.031 INFO:teuthology.orchestra.run.smithi116.stdout:generic/167 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:11.037 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/167 2024-02-18T12:23:11.037 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/167 2024-02-18T12:23:11.037 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:11.040 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:11.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:11.138 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-02-18T12:23:11.434 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:11.438 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:11.439 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:11.440 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:11.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:11.781 INFO:teuthology.orchestra.run.smithi116.stdout:generic/168 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:11.786 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/168 2024-02-18T12:23:11.786 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/168 2024-02-18T12:23:11.786 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:11.790 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:11.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:11.887 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-02-18T12:23:12.160 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:12.164 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:12.164 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:12.165 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:12.165 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:12.561 INFO:teuthology.orchestra.run.smithi116.stdout:generic/169 0s 2024-02-18T12:23:12.567 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/169 2024-02-18T12:23:12.567 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:12.569 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:12.577 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:12.578 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-02-18T12:23:12.760 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:12.763 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:12.764 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:12.765 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:12.765 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:13.092 INFO:teuthology.orchestra.run.smithi116.stdout:generic/170 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:13.098 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/170 2024-02-18T12:23:13.098 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/170 2024-02-18T12:23:13.098 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:13.101 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:13.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:13.150 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-02-18T12:23:13.474 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:13.478 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:13.478 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:13.479 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:13.479 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:13.828 INFO:teuthology.orchestra.run.smithi116.stdout:generic/171 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:13.833 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/171 2024-02-18T12:23:13.833 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/171 2024-02-18T12:23:13.833 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:13.837 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:13.902 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:13.902 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-02-18T12:23:14.197 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:14.201 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:14.201 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:14.202 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:14.202 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:14.530 INFO:teuthology.orchestra.run.smithi116.stdout:generic/172 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:14.543 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/172 2024-02-18T12:23:14.544 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/172 2024-02-18T12:23:14.544 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:14.546 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:14.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:14.611 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-02-18T12:23:14.909 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:14.913 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:14.914 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:14.914 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:14.915 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:15.269 INFO:teuthology.orchestra.run.smithi116.stdout:generic/173 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:15.274 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/173 2024-02-18T12:23:15.274 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/173 2024-02-18T12:23:15.274 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:15.278 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:15.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:15.444 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-02-18T12:23:15.685 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:15.688 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:15.689 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:15.690 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:15.690 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:16.038 INFO:teuthology.orchestra.run.smithi116.stdout:generic/174 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:16.044 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/174 2024-02-18T12:23:16.044 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/174 2024-02-18T12:23:16.044 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:16.047 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:16.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:16.157 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-02-18T12:23:16.432 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:16.435 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:16.436 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:16.437 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:16.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:16.797 INFO:teuthology.orchestra.run.smithi116.stdout:generic/175 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:16.802 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/175 2024-02-18T12:23:16.802 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/175 2024-02-18T12:23:16.802 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:16.806 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:16.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:16.925 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-02-18T12:23:17.183 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:17.184 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:17.184 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:17.185 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:17.186 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:17.534 INFO:teuthology.orchestra.run.smithi116.stdout:generic/176 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:17.540 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/176 2024-02-18T12:23:17.540 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/176 2024-02-18T12:23:17.540 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:17.546 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:17.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:17.634 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-02-18T12:23:17.907 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:17.910 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:17.911 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:17.912 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:17.912 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:18.261 INFO:teuthology.orchestra.run.smithi116.stdout:generic/177 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T12:23:18.265 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/177 2024-02-18T12:23:18.265 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/177 2024-02-18T12:23:18.265 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:18.267 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:18.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:18.309 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-02-18T12:23:18.670 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:18.674 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:18.674 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:18.675 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:18.675 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:18.913 INFO:teuthology.orchestra.run.smithi116.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:18.917 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/178 2024-02-18T12:23:18.917 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/178 2024-02-18T12:23:18.917 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:18.919 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:18.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:18.975 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-02-18T12:23:19.316 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:19.319 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:19.320 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:19.321 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:19.321 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:19.553 INFO:teuthology.orchestra.run.smithi116.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:19.557 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/179 2024-02-18T12:23:19.557 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/179 2024-02-18T12:23:19.557 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:19.559 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:19.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:19.606 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-02-18T12:23:19.960 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:19.964 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:19.964 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:19.965 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:19.965 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:20.194 INFO:teuthology.orchestra.run.smithi116.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:20.198 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/180 2024-02-18T12:23:20.198 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/180 2024-02-18T12:23:20.198 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:20.200 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:20.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:20.253 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-02-18T12:23:20.598 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:20.602 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:20.603 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:20.604 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:20.604 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:20.830 INFO:teuthology.orchestra.run.smithi116.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:23:20.834 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/181 2024-02-18T12:23:20.834 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/181 2024-02-18T12:23:20.834 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:20.836 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:20.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:20.896 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-02-18T12:23:21.226 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:21.542 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:21.542 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/182 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/182 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:21.543 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:21.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:21.544 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-02-18T12:23:21.881 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:21.884 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:21.885 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:21.886 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:21.886 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:22.282 INFO:teuthology.orchestra.run.smithi116.stdout:generic/183 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:22.288 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/183 2024-02-18T12:23:22.288 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/183 2024-02-18T12:23:22.288 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:22.290 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:22.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:22.351 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-02-18T12:23:22.656 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:22.660 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:22.660 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:22.661 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:22.662 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:22.915 INFO:teuthology.orchestra.run.smithi116.stdout:generic/184 0s 2024-02-18T12:23:22.920 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/184 2024-02-18T12:23:22.920 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:22.923 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:22.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:22.936 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-02-18T12:23:23.114 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:23.118 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:23.118 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:23.119 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:23.119 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:23.468 INFO:teuthology.orchestra.run.smithi116.stdout:generic/185 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:23.473 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/185 2024-02-18T12:23:23.473 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/185 2024-02-18T12:23:23.473 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:23.477 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:23.655 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:23.655 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-02-18T12:23:23.867 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:23.871 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:23.871 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:23.872 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:23.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:24.208 INFO:teuthology.orchestra.run.smithi116.stdout:generic/186 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:24.218 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/186 2024-02-18T12:23:24.218 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/186 2024-02-18T12:23:24.218 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:24.221 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:24.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:24.378 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-02-18T12:23:24.619 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:24.623 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:24.623 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:24.624 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:24.625 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:24.952 INFO:teuthology.orchestra.run.smithi116.stdout:generic/187 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:24.959 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/187 2024-02-18T12:23:24.959 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/187 2024-02-18T12:23:24.959 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:24.961 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:25.083 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:25.083 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-02-18T12:23:25.344 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:25.348 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:25.348 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:25.349 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:25.350 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:25.674 INFO:teuthology.orchestra.run.smithi116.stdout:generic/188 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:25.679 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/188 2024-02-18T12:23:25.679 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/188 2024-02-18T12:23:25.679 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:25.687 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:25.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:25.762 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-02-18T12:23:26.070 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:26.074 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:26.075 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:26.076 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:26.076 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:26.407 INFO:teuthology.orchestra.run.smithi116.stdout:generic/189 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:26.418 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/189 2024-02-18T12:23:26.418 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/189 2024-02-18T12:23:26.418 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:26.420 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:26.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:26.652 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-02-18T12:23:26.850 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:26.854 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:26.854 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:26.855 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:26.855 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:27.188 INFO:teuthology.orchestra.run.smithi116.stdout:generic/190 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:27.193 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/190 2024-02-18T12:23:27.193 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/190 2024-02-18T12:23:27.193 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:27.197 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:27.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:27.257 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-02-18T12:23:27.559 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:27.563 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:27.563 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:27.564 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:27.564 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:27.887 INFO:teuthology.orchestra.run.smithi116.stdout:generic/191 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:27.894 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/191 2024-02-18T12:23:27.894 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/191 2024-02-18T12:23:27.895 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:27.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:27.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:27.961 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-02-18T12:23:28.271 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:28.274 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:28.275 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:28.276 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:28.276 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:28.557 INFO:teuthology.orchestra.run.smithi116.stdout:generic/192 [not run] atime not maintained by ceph 2024-02-18T12:23:28.560 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/192 2024-02-18T12:23:28.560 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/192 2024-02-18T12:23:28.561 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:28.574 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:28.574 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-02-18T12:23:28.761 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:28.764 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:28.765 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:28.766 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:28.766 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:31.385 INFO:teuthology.orchestra.run.smithi116.stdout:generic/193 3s 2024-02-18T12:23:31.391 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/193 2024-02-18T12:23:31.391 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:31.393 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:31.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:31.448 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-02-18T12:23:31.801 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:31.805 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:31.806 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:31.807 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:31.807 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:32.138 INFO:teuthology.orchestra.run.smithi116.stdout:generic/194 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:32.147 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/194 2024-02-18T12:23:32.147 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/194 2024-02-18T12:23:32.147 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:32.149 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:32.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:32.234 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-02-18T12:23:32.508 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:32.512 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:32.512 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:32.513 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:32.514 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:32.864 INFO:teuthology.orchestra.run.smithi116.stdout:generic/195 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:32.871 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/195 2024-02-18T12:23:32.872 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/195 2024-02-18T12:23:32.872 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:32.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:32.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:32.967 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-02-18T12:23:33.250 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:33.254 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:33.254 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:33.256 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:33.256 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:33.600 INFO:teuthology.orchestra.run.smithi116.stdout:generic/196 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:33.605 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/196 2024-02-18T12:23:33.605 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/196 2024-02-18T12:23:33.606 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:33.608 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:33.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:33.773 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-02-18T12:23:34.005 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:34.009 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:34.010 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:34.011 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:34.011 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:34.345 INFO:teuthology.orchestra.run.smithi116.stdout:generic/197 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:34.352 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/197 2024-02-18T12:23:34.352 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/197 2024-02-18T12:23:34.352 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:34.356 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:34.471 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:34.471 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-02-18T12:23:34.748 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:34.752 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:34.752 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:34.754 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:34.754 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:35.391 INFO:teuthology.orchestra.run.smithi116.stdout:generic/198 1s 2024-02-18T12:23:35.397 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/198 2024-02-18T12:23:35.397 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:35.399 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:35.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:35.465 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-02-18T12:23:35.820 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:35.821 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:35.821 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:35.822 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:35.822 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:36.158 INFO:teuthology.orchestra.run.smithi116.stdout:generic/199 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:36.166 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/199 2024-02-18T12:23:36.166 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/199 2024-02-18T12:23:36.166 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:36.167 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:36.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:36.218 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-02-18T12:23:36.542 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:36.547 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:36.547 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:36.549 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:36.549 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:generic/200 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:36.917 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/200 2024-02-18T12:23:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/200 2024-02-18T12:23:36.918 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:36.920 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:37.121 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:37.122 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-02-18T12:23:37.332 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:37.336 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:37.336 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:37.337 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:37.337 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:37.663 INFO:teuthology.orchestra.run.smithi116.stdout:generic/201 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:37.670 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/201 2024-02-18T12:23:37.670 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/201 2024-02-18T12:23:37.671 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:37.672 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:37.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:37.824 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-02-18T12:23:38.071 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:38.075 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:38.076 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:38.076 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:38.077 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:38.451 INFO:teuthology.orchestra.run.smithi116.stdout:generic/202 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:38.457 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/202 2024-02-18T12:23:38.458 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/202 2024-02-18T12:23:38.458 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:38.461 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:38.537 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:38.538 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-02-18T12:23:38.865 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:38.869 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:38.869 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:38.870 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:38.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:39.199 INFO:teuthology.orchestra.run.smithi116.stdout:generic/203 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:39.206 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/203 2024-02-18T12:23:39.206 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/203 2024-02-18T12:23:39.206 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:39.208 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:39.374 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:39.374 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-02-18T12:23:39.618 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:39.621 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:39.621 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:39.622 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:39.622 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:41.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:23:41.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:23:41.215 INFO:teuthology.orchestra.run.smithi116.stdout:generic/204 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:23:41.220 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/204 2024-02-18T12:23:41.220 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/204 2024-02-18T12:23:41.220 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:41.223 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:41.234 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T12:23:41.231+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49815) UID: 0 2024-02-18T12:23:41.234 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T12:23:41.231+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49815) UID: 0 2024-02-18T12:23:41.235 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T12:23:41.231+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49815) UID: 0 2024-02-18T12:23:41.235 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T12:23:41.231+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 49815) UID: 0 2024-02-18T12:23:41.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:41.248 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-02-18T12:23:41.436 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:41.439 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:41.440 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:41.441 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:41.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:41.786 INFO:teuthology.orchestra.run.smithi116.stdout:generic/205 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:41.793 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/205 2024-02-18T12:23:41.793 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/205 2024-02-18T12:23:41.793 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:41.795 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:41.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:41.969 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-02-18T12:23:42.198 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:42.202 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:42.202 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:42.203 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:42.203 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:42.531 INFO:teuthology.orchestra.run.smithi116.stdout:generic/206 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:23:42.536 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/206 2024-02-18T12:23:42.537 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/206 2024-02-18T12:23:42.537 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:42.540 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:42.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:42.683 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-02-18T12:23:42.945 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:42.949 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:42.950 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:42.950 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:42.951 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:50.554 INFO:teuthology.orchestra.run.smithi116.stdout:generic/207 7s 2024-02-18T12:23:50.560 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/207 2024-02-18T12:23:50.560 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:23:50.562 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:23:50.607 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:23:50.608 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-02-18T12:23:50.951 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:23:50.956 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:23:50.956 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:23:50.957 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:23:50.957 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:26:43.648 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:26:43.652 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:27:11.337 INFO:teuthology.orchestra.run.smithi116.stdout:generic/208 200s 2024-02-18T12:27:11.342 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/208 2024-02-18T12:27:11.342 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:11.344 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:11.386 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:11.387 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-02-18T12:27:11.732 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:11.737 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:11.737 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:11.738 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:11.738 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:14.177 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:27:14.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:27:42.823 INFO:teuthology.orchestra.run.smithi116.stdout:generic/209 31s 2024-02-18T12:27:42.828 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/209 2024-02-18T12:27:42.828 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:42.830 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:42.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:42.869 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-02-18T12:27:43.219 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:43.222 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:43.223 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:43.224 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:43.224 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:43.745 INFO:teuthology.orchestra.run.smithi116.stdout:generic/210 0s 2024-02-18T12:27:43.784 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/210 2024-02-18T12:27:43.785 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:43.787 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:43.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:43.830 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-02-18T12:27:44.179 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:44.183 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:44.184 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:44.185 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:44.185 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:44.472 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:27:44.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:27:44.551 INFO:teuthology.orchestra.run.smithi116.stdout:generic/211 0s 2024-02-18T12:27:44.556 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/211 2024-02-18T12:27:44.556 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:44.558 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:44.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:44.573 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-02-18T12:27:44.785 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:44.789 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:44.789 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:44.790 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:44.790 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:45.067 INFO:teuthology.orchestra.run.smithi116.stdout:generic/212 1s 2024-02-18T12:27:45.072 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/212 2024-02-18T12:27:45.072 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:45.074 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:45.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:45.090 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-02-18T12:27:45.268 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:45.272 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:45.272 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:45.273 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:45.273 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:45.483 INFO:teuthology.orchestra.run.smithi116.stdout:generic/213 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:27:45.487 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/213 2024-02-18T12:27:45.487 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/213 2024-02-18T12:27:45.487 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:45.489 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:45.501 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:45.501 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-02-18T12:27:45.678 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:45.682 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:45.682 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:45.683 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:45.683 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:45.904 INFO:teuthology.orchestra.run.smithi116.stdout:generic/214 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:27:45.908 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/214 2024-02-18T12:27:45.908 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/214 2024-02-18T12:27:45.908 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:45.910 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:45.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:45.922 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-02-18T12:27:46.095 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:46.098 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:46.099 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:46.100 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:46.100 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:48.376 INFO:teuthology.orchestra.run.smithi116.stdout:generic/215 2s 2024-02-18T12:27:48.382 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/215 2024-02-18T12:27:48.382 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:48.385 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:48.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:48.417 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-02-18T12:27:48.597 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:48.601 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:48.601 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:48.602 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:48.602 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:48.951 INFO:teuthology.orchestra.run.smithi116.stdout:generic/216 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:48.957 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/216 2024-02-18T12:27:48.957 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/216 2024-02-18T12:27:48.957 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:48.965 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:49.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:49.112 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-02-18T12:27:49.351 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:49.354 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:49.355 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:49.356 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:49.356 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:49.689 INFO:teuthology.orchestra.run.smithi116.stdout:generic/217 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:49.696 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/217 2024-02-18T12:27:49.696 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/217 2024-02-18T12:27:49.696 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:49.699 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:49.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:49.882 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-02-18T12:27:50.114 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:50.118 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:50.118 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:50.119 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:50.119 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:50.460 INFO:teuthology.orchestra.run.smithi116.stdout:generic/218 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:50.469 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/218 2024-02-18T12:27:50.469 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/218 2024-02-18T12:27:50.469 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:50.472 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:50.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:50.623 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-02-18T12:27:50.869 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:50.873 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:50.873 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:50.874 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:50.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:51.077 INFO:teuthology.orchestra.run.smithi116.stdout:generic/219 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:27:51.081 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/219 2024-02-18T12:27:51.081 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/219 2024-02-18T12:27:51.081 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:51.083 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:51.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:51.094 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-02-18T12:27:51.264 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:51.268 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:51.268 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:51.269 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:51.269 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:51.595 INFO:teuthology.orchestra.run.smithi116.stdout:generic/220 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:51.600 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/220 2024-02-18T12:27:51.600 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/220 2024-02-18T12:27:51.600 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:51.602 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:51.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:51.681 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-02-18T12:27:51.971 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:51.974 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:51.975 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:51.976 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:53.221 INFO:teuthology.orchestra.run.smithi116.stdout:generic/221 1s 2024-02-18T12:27:53.227 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/221 2024-02-18T12:27:53.227 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:53.230 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:53.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:53.246 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-02-18T12:27:53.429 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:53.432 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:53.433 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:53.434 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:53.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:53.776 INFO:teuthology.orchestra.run.smithi116.stdout:generic/222 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:53.781 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/222 2024-02-18T12:27:53.781 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/222 2024-02-18T12:27:53.781 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:53.786 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:53.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:53.873 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-02-18T12:27:54.156 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:54.160 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:54.160 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:54.161 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:54.161 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:54.373 INFO:teuthology.orchestra.run.smithi116.stdout:generic/223 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:27:54.377 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/223 2024-02-18T12:27:54.377 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/223 2024-02-18T12:27:54.377 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:54.379 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:54.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:54.407 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-02-18T12:27:54.574 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:54.578 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:54.578 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:54.579 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:54.579 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:54.781 INFO:teuthology.orchestra.run.smithi116.stdout:generic/224 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:27:54.785 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/224 2024-02-18T12:27:54.785 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/224 2024-02-18T12:27:54.785 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:54.787 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:54.798 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:54.799 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-02-18T12:27:54.981 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:54.985 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:54.985 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:54.986 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:54.986 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:55.243 INFO:teuthology.orchestra.run.smithi116.stdout:generic/225 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T12:27:55.247 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/225 2024-02-18T12:27:55.247 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/225 2024-02-18T12:27:55.247 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:55.249 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:55.293 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:55.294 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-02-18T12:27:55.644 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:55.648 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:55.648 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:55.649 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:55.650 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:55.882 INFO:teuthology.orchestra.run.smithi116.stdout:generic/226 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:27:55.886 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/226 2024-02-18T12:27:55.886 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/226 2024-02-18T12:27:55.886 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:55.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:55.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:55.914 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-02-18T12:27:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:56.096 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:56.097 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:56.098 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:56.098 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:56.429 INFO:teuthology.orchestra.run.smithi116.stdout:generic/227 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:56.434 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/227 2024-02-18T12:27:56.434 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/227 2024-02-18T12:27:56.435 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:56.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:56.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:56.624 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-02-18T12:27:56.844 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:56.848 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:56.848 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:56.849 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:56.849 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:57.085 INFO:teuthology.orchestra.run.smithi116.stdout:generic/228 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:27:57.089 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/228 2024-02-18T12:27:57.089 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/228 2024-02-18T12:27:57.089 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:57.091 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:57.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:57.102 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-02-18T12:27:57.278 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:57.282 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:57.282 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:57.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:57.628 INFO:teuthology.orchestra.run.smithi116.stdout:generic/229 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:27:57.634 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/229 2024-02-18T12:27:57.635 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/229 2024-02-18T12:27:57.635 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:57.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:57.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:57.681 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-02-18T12:27:57.983 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:57.987 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:57.987 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:57.988 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:57.988 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:58.193 INFO:teuthology.orchestra.run.smithi116.stdout:generic/230 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:27:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/230 2024-02-18T12:27:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/230 2024-02-18T12:27:58.197 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:58.200 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:58.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:58.210 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-02-18T12:27:58.381 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:58.385 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:58.385 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:58.386 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:58.387 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:58.584 INFO:teuthology.orchestra.run.smithi116.stdout:generic/231 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:27:58.587 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/231 2024-02-18T12:27:58.588 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/231 2024-02-18T12:27:58.588 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:58.590 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:58.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:58.601 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-02-18T12:27:58.786 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:58.789 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:58.790 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:58.791 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:58.791 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:58.986 INFO:teuthology.orchestra.run.smithi116.stdout:generic/232 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:27:58.989 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/232 2024-02-18T12:27:58.989 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/232 2024-02-18T12:27:58.989 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:58.992 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:59.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:59.025 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-02-18T12:27:59.201 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:59.204 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:59.205 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:59.206 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:59.206 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:59.402 INFO:teuthology.orchestra.run.smithi116.stdout:generic/233 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:27:59.405 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/233 2024-02-18T12:27:59.406 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/233 2024-02-18T12:27:59.406 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:59.408 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:59.430 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:59.431 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-02-18T12:27:59.607 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:27:59.611 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:27:59.611 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:27:59.612 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:27:59.612 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:59.809 INFO:teuthology.orchestra.run.smithi116.stdout:generic/234 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:27:59.813 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/234 2024-02-18T12:27:59.813 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/234 2024-02-18T12:27:59.813 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:27:59.815 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:27:59.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:27:59.827 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-02-18T12:28:00.004 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:00.007 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:00.008 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:00.009 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:00.009 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:00.206 INFO:teuthology.orchestra.run.smithi116.stdout:generic/235 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:28:00.209 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/235 2024-02-18T12:28:00.209 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/235 2024-02-18T12:28:00.209 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:28:00.212 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:00.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:28:00.223 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-02-18T12:28:00.401 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:00.405 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:00.405 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:00.406 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:00.406 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:01.685 INFO:teuthology.orchestra.run.smithi116.stdout:generic/236 1s 2024-02-18T12:28:01.691 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/236 2024-02-18T12:28:01.691 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:28:01.694 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:01.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:28:01.748 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-02-18T12:28:02.084 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:02.087 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:02.088 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:02.089 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:02.089 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:02.375 INFO:teuthology.orchestra.run.smithi116.stdout:generic/237 0s 2024-02-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/237 2024-02-18T12:28:02.380 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:28:02.382 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:02.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:28:02.413 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-02-18T12:28:02.788 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:02.792 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:02.792 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:02.793 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:02.793 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:03.119 INFO:teuthology.orchestra.run.smithi116.stdout:generic/238 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:28:03.126 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/238 2024-02-18T12:28:03.126 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/238 2024-02-18T12:28:03.126 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:28:03.128 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:03.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:28:03.219 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-02-18T12:28:03.518 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:03.522 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:03.522 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:03.523 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:03.523 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:03.751 INFO:teuthology.orchestra.run.smithi116.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-02-18T12:28:03.755 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/239 2024-02-18T12:28:03.755 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/239 2024-02-18T12:28:03.755 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:28:03.758 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:03.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:28:03.778 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-02-18T12:28:03.947 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:03.951 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:03.952 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:04.174 INFO:teuthology.orchestra.run.smithi116.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-02-18T12:28:04.178 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/240 2024-02-18T12:28:04.178 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/240 2024-02-18T12:28:04.178 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:28:04.180 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:04.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:28:04.201 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-02-18T12:28:04.377 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:28:04.381 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:28:04.381 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:28:04.382 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:28:04.382 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:28:14.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:28:14.554 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:28:44.632 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:28:44.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:29:14.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:29:14.741 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:29:18.724 INFO:teuthology.orchestra.run.smithi116.stdout:generic/241 74s 2024-02-18T12:29:18.748 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/241 2024-02-18T12:29:18.749 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:18.751 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:18.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:18.915 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-02-18T12:29:19.602 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:19.609 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:19.609 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:19.611 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:19.611 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:20.329 INFO:teuthology.orchestra.run.smithi116.stdout:generic/242 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:20.341 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/242 2024-02-18T12:29:20.341 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/242 2024-02-18T12:29:20.341 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:20.341 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:20.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:20.558 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-02-18T12:29:20.804 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:20.808 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:20.808 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:20.809 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:20.809 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:21.181 INFO:teuthology.orchestra.run.smithi116.stdout:generic/243 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:21.186 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/243 2024-02-18T12:29:21.186 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/243 2024-02-18T12:29:21.187 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:21.191 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:21.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:21.374 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-02-18T12:29:21.574 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:21.578 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:21.579 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:21.580 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:21.580 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:21.785 INFO:teuthology.orchestra.run.smithi116.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:29:21.789 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/244 2024-02-18T12:29:21.789 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/244 2024-02-18T12:29:21.789 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:21.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:21.803 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-02-18T12:29:21.975 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:21.978 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:21.979 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:21.980 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:21.980 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:22.237 INFO:teuthology.orchestra.run.smithi116.stdout:generic/245 0s 2024-02-18T12:29:22.243 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/245 2024-02-18T12:29:22.243 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:22.245 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:22.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:22.301 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-02-18T12:29:22.644 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:22.647 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:22.648 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:22.649 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:22.649 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:22.916 INFO:teuthology.orchestra.run.smithi116.stdout:generic/246 0s 2024-02-18T12:29:22.921 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/246 2024-02-18T12:29:22.922 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:22.924 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:22.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:22.979 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-02-18T12:29:23.343 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:23.346 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:23.347 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:23.348 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:23.348 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:30.745 INFO:teuthology.orchestra.run.smithi116.stdout:generic/247 7s 2024-02-18T12:29:30.750 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/247 2024-02-18T12:29:30.750 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:30.752 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:30.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:30.761 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-02-18T12:29:30.939 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:30.942 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:30.943 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:30.944 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:30.944 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:31.188 INFO:teuthology.orchestra.run.smithi116.stdout:generic/248 0s 2024-02-18T12:29:31.193 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/248 2024-02-18T12:29:31.193 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:31.195 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:31.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:31.217 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-02-18T12:29:31.390 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:31.394 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:31.394 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:31.395 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:31.395 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:31.816 INFO:teuthology.orchestra.run.smithi116.stdout:generic/249 0s 2024-02-18T12:29:31.821 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/249 2024-02-18T12:29:31.821 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:31.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:31.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:31.874 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-02-18T12:29:32.230 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:32.234 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:32.235 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:32.236 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:32.236 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:32.479 INFO:teuthology.orchestra.run.smithi116.stdout:generic/250 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:29:32.482 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/250 2024-02-18T12:29:32.483 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/250 2024-02-18T12:29:32.483 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:32.485 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:32.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:32.497 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-02-18T12:29:32.671 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:32.674 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:32.675 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:32.676 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:32.676 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:32.975 INFO:teuthology.orchestra.run.smithi116.stdout:generic/251 [not run] FITRIM not supported on /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:32.983 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/251 2024-02-18T12:29:32.983 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/251 2024-02-18T12:29:32.983 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:32.985 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:32.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:32.997 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-02-18T12:29:33.172 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:33.176 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:33.176 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:33.177 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:33.177 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:33.388 INFO:teuthology.orchestra.run.smithi116.stdout:generic/252 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:29:33.392 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/252 2024-02-18T12:29:33.392 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/252 2024-02-18T12:29:33.392 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:33.394 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:33.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:33.406 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-02-18T12:29:33.593 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:33.596 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:33.597 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:33.598 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:33.598 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:33.929 INFO:teuthology.orchestra.run.smithi116.stdout:generic/253 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:33.934 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/253 2024-02-18T12:29:33.935 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/253 2024-02-18T12:29:33.935 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:33.943 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:34.175 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:34.176 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-02-18T12:29:34.380 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:34.383 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:34.384 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:34.385 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:34.385 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:34.730 INFO:teuthology.orchestra.run.smithi116.stdout:generic/254 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:34.736 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/254 2024-02-18T12:29:34.736 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/254 2024-02-18T12:29:34.736 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:34.738 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:34.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:34.960 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-02-18T12:29:35.155 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:35.159 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:35.159 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:35.160 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:35.160 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:35.472 INFO:teuthology.orchestra.run.smithi116.stdout:generic/255 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:29:35.476 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/255 2024-02-18T12:29:35.476 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/255 2024-02-18T12:29:35.477 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:35.479 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:35.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:35.531 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-02-18T12:29:35.886 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:35.890 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:35.890 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:35.891 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:35.891 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:36.519 INFO:teuthology.orchestra.run.smithi116.stdout:generic/256 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:29:36.523 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/256 2024-02-18T12:29:36.523 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/256 2024-02-18T12:29:36.523 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:36.525 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:36.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:36.543 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-02-18T12:29:36.721 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:36.725 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:36.725 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:36.726 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:36.726 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:37.316 INFO:teuthology.orchestra.run.smithi116.stdout:generic/257 1s 2024-02-18T12:29:37.321 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/257 2024-02-18T12:29:37.322 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:37.324 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:37.447 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:37.447 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-02-18T12:29:37.704 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:37.708 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:37.708 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:37.709 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:37.709 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:37.978 INFO:teuthology.orchestra.run.smithi116.stdout:generic/258 [not run] ceph does not support negative timestamps 2024-02-18T12:29:37.985 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/258 2024-02-18T12:29:37.985 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/258 2024-02-18T12:29:37.985 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:37.987 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:38.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:38.017 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-02-18T12:29:38.193 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:38.196 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:38.197 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:38.198 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:38.198 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:38.541 INFO:teuthology.orchestra.run.smithi116.stdout:generic/259 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:38.547 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/259 2024-02-18T12:29:38.547 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/259 2024-02-18T12:29:38.547 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:38.550 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:38.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:38.785 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-02-18T12:29:38.981 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:38.984 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:38.985 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:38.986 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:38.986 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:39.252 INFO:teuthology.orchestra.run.smithi116.stdout:generic/260 [not run] FITRIM not supported on /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:39.259 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/260 2024-02-18T12:29:39.260 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/260 2024-02-18T12:29:39.260 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:39.262 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:39.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:39.284 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-02-18T12:29:39.465 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:39.469 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:39.469 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:39.470 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:39.470 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:39.804 INFO:teuthology.orchestra.run.smithi116.stdout:generic/261 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:39.809 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/261 2024-02-18T12:29:39.809 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/261 2024-02-18T12:29:39.809 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:39.815 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:40.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:40.065 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-02-18T12:29:40.247 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:40.250 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:40.251 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:40.252 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:40.252 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:40.597 INFO:teuthology.orchestra.run.smithi116.stdout:generic/262 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:29:40.602 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/262 2024-02-18T12:29:40.603 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/262 2024-02-18T12:29:40.603 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:29:40.606 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:40.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:29:40.807 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-02-18T12:29:41.010 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:29:41.014 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:29:41.015 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:29:41.016 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:29:41.016 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:29:44.946 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:29:44.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:30:15.007 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:30:15.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:30:19.460 INFO:teuthology.orchestra.run.smithi116.stdout:generic/263 38s 2024-02-18T12:30:19.466 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/263 2024-02-18T12:30:19.466 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:19.468 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:19.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:19.509 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-02-18T12:30:19.898 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:19.902 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:19.902 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:19.903 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:19.903 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:20.250 INFO:teuthology.orchestra.run.smithi116.stdout:generic/264 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:20.256 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/264 2024-02-18T12:30:20.260 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/264 2024-02-18T12:30:20.260 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:20.264 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:20.590 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:20.590 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-02-18T12:30:20.776 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:20.777 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:20.777 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:20.778 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:20.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:21.148 INFO:teuthology.orchestra.run.smithi116.stdout:generic/265 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:21.153 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/265 2024-02-18T12:30:21.153 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/265 2024-02-18T12:30:21.153 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:21.157 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:21.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:21.453 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-02-18T12:30:21.633 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:21.637 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:21.637 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:21.638 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:21.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:22.043 INFO:teuthology.orchestra.run.smithi116.stdout:generic/266 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:22.050 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/266 2024-02-18T12:30:22.051 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/266 2024-02-18T12:30:22.051 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:22.054 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:22.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:22.302 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-02-18T12:30:22.479 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:22.482 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:22.483 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:22.484 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:22.484 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:22.844 INFO:teuthology.orchestra.run.smithi116.stdout:generic/267 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:22.852 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/267 2024-02-18T12:30:22.853 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/267 2024-02-18T12:30:22.853 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:22.856 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:23.114 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:23.114 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-02-18T12:30:23.295 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:23.298 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:23.298 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:23.299 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:23.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:23.663 INFO:teuthology.orchestra.run.smithi116.stdout:generic/268 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:23.673 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/268 2024-02-18T12:30:23.674 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/268 2024-02-18T12:30:23.674 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:23.676 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:23.927 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:23.927 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-02-18T12:30:24.109 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:24.113 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:24.113 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:24.114 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:24.114 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:24.323 INFO:teuthology.orchestra.run.smithi116.stdout:generic/269 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:30:24.326 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/269 2024-02-18T12:30:24.327 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/269 2024-02-18T12:30:24.327 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:24.328 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:24.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:24.358 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-02-18T12:30:24.537 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:24.540 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:24.541 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:24.541 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:24.542 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:24.746 INFO:teuthology.orchestra.run.smithi116.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:30:24.750 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/270 2024-02-18T12:30:24.750 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/270 2024-02-18T12:30:24.750 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:24.752 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:24.765 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:24.765 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-02-18T12:30:24.948 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:24.951 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:24.952 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:24.953 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:24.953 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:25.307 INFO:teuthology.orchestra.run.smithi116.stdout:generic/271 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:25.317 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/271 2024-02-18T12:30:25.317 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/271 2024-02-18T12:30:25.317 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:25.320 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:25.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:25.592 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-02-18T12:30:25.783 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:25.787 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:25.787 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:25.788 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:25.788 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:26.165 INFO:teuthology.orchestra.run.smithi116.stdout:generic/272 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:26.172 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/272 2024-02-18T12:30:26.172 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/272 2024-02-18T12:30:26.172 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:26.173 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:26.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:26.437 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-02-18T12:30:26.625 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:26.628 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:26.628 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:26.629 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:26.630 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:26.836 INFO:teuthology.orchestra.run.smithi116.stdout:generic/273 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:30:26.840 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/273 2024-02-18T12:30:26.840 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/273 2024-02-18T12:30:26.840 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:26.843 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:26.866 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:26.866 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-02-18T12:30:27.042 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:27.046 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:27.046 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:27.048 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:27.048 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:27.259 INFO:teuthology.orchestra.run.smithi116.stdout:generic/274 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T12:30:27.263 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/274 2024-02-18T12:30:27.263 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/274 2024-02-18T12:30:27.263 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:27.265 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:27.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:27.282 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-02-18T12:30:27.456 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:27.459 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:27.460 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:27.461 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:27.461 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:27.667 INFO:teuthology.orchestra.run.smithi116.stdout:generic/275 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:30:27.671 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/275 2024-02-18T12:30:27.671 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/275 2024-02-18T12:30:27.671 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:27.673 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:27.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:27.686 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-02-18T12:30:27.865 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:27.869 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:27.869 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:27.870 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:27.871 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:28.226 INFO:teuthology.orchestra.run.smithi116.stdout:generic/276 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:28.234 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/276 2024-02-18T12:30:28.234 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/276 2024-02-18T12:30:28.234 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:28.236 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:28.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:28.481 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-02-18T12:30:28.672 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:28.675 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:28.676 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:28.677 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:28.677 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:28.936 INFO:teuthology.orchestra.run.smithi116.stdout:generic/277 [not run] file system doesn't support chattr +A 2024-02-18T12:30:28.940 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/277 2024-02-18T12:30:28.940 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/277 2024-02-18T12:30:28.941 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:28.943 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:28.970 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:28.971 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-02-18T12:30:29.146 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:29.150 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:29.151 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:29.151 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:29.152 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:29.521 INFO:teuthology.orchestra.run.smithi116.stdout:generic/278 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:29.527 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/278 2024-02-18T12:30:29.527 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/278 2024-02-18T12:30:29.528 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:29.529 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:29.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:29.789 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-02-18T12:30:29.977 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:29.981 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:29.981 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:29.982 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:29.983 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:30.346 INFO:teuthology.orchestra.run.smithi116.stdout:generic/279 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:30.351 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/279 2024-02-18T12:30:30.351 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/279 2024-02-18T12:30:30.351 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:30.354 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:30.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:30.634 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-02-18T12:30:30.825 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:30.829 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:30.829 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:30.830 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:30.830 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:31.164 INFO:teuthology.orchestra.run.smithi116.stdout:generic/280 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:30:31.168 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/280 2024-02-18T12:30:31.168 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/280 2024-02-18T12:30:31.168 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:31.170 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:31.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:31.183 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-02-18T12:30:31.356 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:31.360 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:31.360 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:31.361 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:31.361 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:31.817 INFO:teuthology.orchestra.run.smithi116.stdout:generic/281 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:31.824 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/281 2024-02-18T12:30:31.824 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/281 2024-02-18T12:30:31.824 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:31.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:31.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:31.999 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-02-18T12:30:32.207 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:32.211 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:32.211 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:32.212 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:32.212 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:32.577 INFO:teuthology.orchestra.run.smithi116.stdout:generic/282 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:32.582 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/282 2024-02-18T12:30:32.583 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/282 2024-02-18T12:30:32.583 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:32.586 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:32.836 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:32.836 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-02-18T12:30:33.016 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:33.020 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:33.020 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:33.021 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:33.021 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:33.380 INFO:teuthology.orchestra.run.smithi116.stdout:generic/283 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:33.388 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/283 2024-02-18T12:30:33.388 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/283 2024-02-18T12:30:33.389 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:33.392 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:33.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:33.678 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-02-18T12:30:33.859 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:33.863 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:33.863 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:33.864 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:33.864 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:34.229 INFO:teuthology.orchestra.run.smithi116.stdout:generic/284 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:34.234 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/284 2024-02-18T12:30:34.234 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/284 2024-02-18T12:30:34.234 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:34.239 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:34.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:34.500 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-02-18T12:30:34.684 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:34.688 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:34.688 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:34.689 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:34.689 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:35.939 INFO:teuthology.orchestra.run.smithi116.stdout:generic/285 1s 2024-02-18T12:30:35.945 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/285 2024-02-18T12:30:35.945 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:35.947 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:36.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:36.337 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-02-18T12:30:36.711 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:36.715 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:36.715 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:36.716 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:36.716 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:36.975 INFO:teuthology.orchestra.run.smithi116.stdout:generic/286 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:30:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/286 2024-02-18T12:30:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/286 2024-02-18T12:30:36.987 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:36.991 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:37.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:37.013 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-02-18T12:30:37.234 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:37.239 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:37.239 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:37.240 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:37.241 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:37.661 INFO:teuthology.orchestra.run.smithi116.stdout:generic/287 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:37.668 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/287 2024-02-18T12:30:37.668 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/287 2024-02-18T12:30:37.668 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:37.670 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:37.983 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:37.983 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-02-18T12:30:38.216 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:38.221 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:38.222 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:38.223 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:38.223 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:38.612 INFO:teuthology.orchestra.run.smithi116.stdout:generic/288 [not run] FITRIM not supported on /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:38.637 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/288 2024-02-18T12:30:38.637 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/288 2024-02-18T12:30:38.637 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:38.640 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:38.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:38.659 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-02-18T12:30:38.897 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:38.902 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:38.903 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:38.904 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:38.904 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:39.327 INFO:teuthology.orchestra.run.smithi116.stdout:generic/289 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:39.333 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/289 2024-02-18T12:30:39.334 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/289 2024-02-18T12:30:39.334 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:39.340 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:39.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:39.617 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-02-18T12:30:39.840 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:39.844 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:39.845 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:39.846 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:39.846 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:40.272 INFO:teuthology.orchestra.run.smithi116.stdout:generic/290 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:40.278 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/290 2024-02-18T12:30:40.278 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/290 2024-02-18T12:30:40.278 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:40.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:40.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:40.565 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-02-18T12:30:40.812 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:40.817 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:40.818 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:40.819 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:40.819 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:41.279 INFO:teuthology.orchestra.run.smithi116.stdout:generic/291 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:41.286 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/291 2024-02-18T12:30:41.286 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/291 2024-02-18T12:30:41.286 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:41.288 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:41.571 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:41.571 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-02-18T12:30:41.808 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:41.812 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:41.812 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:41.813 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:41.813 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:42.259 INFO:teuthology.orchestra.run.smithi116.stdout:generic/292 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:42.264 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/292 2024-02-18T12:30:42.264 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/292 2024-02-18T12:30:42.264 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:42.267 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:42.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:42.557 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-02-18T12:30:42.732 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:42.736 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:42.736 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:42.737 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:42.737 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:43.086 INFO:teuthology.orchestra.run.smithi116.stdout:generic/293 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:43.093 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/293 2024-02-18T12:30:43.093 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/293 2024-02-18T12:30:43.093 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:43.098 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:43.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:43.365 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-02-18T12:30:43.543 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:43.547 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:43.547 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:43.548 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:43.549 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:44.000 INFO:teuthology.orchestra.run.smithi116.stdout:generic/294 0s 2024-02-18T12:30:44.012 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/294 2024-02-18T12:30:44.012 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:44.015 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:44.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:44.031 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-02-18T12:30:44.346 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:44.350 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:44.350 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:44.351 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:44.351 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:44.716 INFO:teuthology.orchestra.run.smithi116.stdout:generic/295 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:44.721 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/295 2024-02-18T12:30:44.721 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/295 2024-02-18T12:30:44.721 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:44.724 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:45.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:45.014 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-02-18T12:30:45.106 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:30:45.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:30:45.193 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:45.197 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:45.197 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:45.198 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:45.198 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:45.568 INFO:teuthology.orchestra.run.smithi116.stdout:generic/296 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:45.573 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/296 2024-02-18T12:30:45.573 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/296 2024-02-18T12:30:45.573 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:45.576 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:45.829 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:45.829 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-02-18T12:30:46.049 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:46.053 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:46.053 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:46.054 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:46.055 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:46.433 INFO:teuthology.orchestra.run.smithi116.stdout:generic/297 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:46.438 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/297 2024-02-18T12:30:46.439 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/297 2024-02-18T12:30:46.439 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:46.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:46.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:46.692 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-02-18T12:30:46.878 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:46.882 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:46.882 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:46.883 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:46.883 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:47.245 INFO:teuthology.orchestra.run.smithi116.stdout:generic/298 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:47.254 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/298 2024-02-18T12:30:47.254 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/298 2024-02-18T12:30:47.254 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:47.259 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:47.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:47.524 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-02-18T12:30:47.722 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:47.726 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:47.726 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:47.727 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:47.727 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:48.107 INFO:teuthology.orchestra.run.smithi116.stdout:generic/299 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:30:48.111 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/299 2024-02-18T12:30:48.111 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/299 2024-02-18T12:30:48.111 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:48.113 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:48.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:48.133 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-02-18T12:30:48.307 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:48.311 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:48.311 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:48.312 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:48.312 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:48.597 INFO:teuthology.orchestra.run.smithi116.stdout:generic/300 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:30:48.601 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/300 2024-02-18T12:30:48.601 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/300 2024-02-18T12:30:48.601 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:48.603 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:48.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:48.624 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-02-18T12:30:48.798 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:48.801 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:48.802 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:48.803 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:48.803 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:49.160 INFO:teuthology.orchestra.run.smithi116.stdout:generic/301 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:49.165 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/301 2024-02-18T12:30:49.165 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/301 2024-02-18T12:30:49.165 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:49.168 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:49.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:49.450 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-02-18T12:30:49.632 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:49.636 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:49.636 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:49.637 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:49.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:50.018 INFO:teuthology.orchestra.run.smithi116.stdout:generic/302 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:50.024 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/302 2024-02-18T12:30:50.024 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/302 2024-02-18T12:30:50.024 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:50.029 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:50.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:50.295 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-02-18T12:30:50.483 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:50.487 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:50.487 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:50.488 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:50.489 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:50.741 INFO:teuthology.orchestra.run.smithi116.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:30:50.745 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/303 2024-02-18T12:30:50.745 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/303 2024-02-18T12:30:50.745 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:50.748 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:50.799 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:50.800 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-02-18T12:30:51.198 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:51.202 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:51.202 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:51.203 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:51.203 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:51.451 INFO:teuthology.orchestra.run.smithi116.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:30:51.455 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/304 2024-02-18T12:30:51.455 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/304 2024-02-18T12:30:51.455 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:51.457 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:51.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:51.517 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-02-18T12:30:51.897 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:51.900 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:51.901 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:51.902 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:51.902 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:52.251 INFO:teuthology.orchestra.run.smithi116.stdout:generic/305 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:30:52.257 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/305 2024-02-18T12:30:52.257 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/305 2024-02-18T12:30:52.257 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:52.260 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:52.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:52.532 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-02-18T12:30:52.728 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:52.732 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:52.732 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:52.733 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:52.733 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:53.260 INFO:teuthology.orchestra.run.smithi116.stdout:generic/306 1s 2024-02-18T12:30:53.270 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/306 2024-02-18T12:30:53.270 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:53.274 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:53.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:53.522 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-02-18T12:30:53.903 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:53.908 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:53.908 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:53.909 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:53.909 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:55.378 INFO:teuthology.orchestra.run.smithi116.stdout:generic/307 1s 2024-02-18T12:30:55.383 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/307 2024-02-18T12:30:55.383 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:55.386 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:55.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:55.395 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-02-18T12:30:55.570 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:55.574 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:55.574 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:55.575 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:55.575 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:55.818 INFO:teuthology.orchestra.run.smithi116.stdout:generic/308 0s 2024-02-18T12:30:55.823 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/308 2024-02-18T12:30:55.823 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:55.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:55.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:55.843 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-02-18T12:30:56.021 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:56.025 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:56.026 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:56.027 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:56.027 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:57.291 INFO:teuthology.orchestra.run.smithi116.stdout:generic/309 1s 2024-02-18T12:30:57.297 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/309 2024-02-18T12:30:57.297 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:30:57.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:30:57.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:30:57.351 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-02-18T12:30:57.712 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:30:57.716 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:30:57.716 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:30:57.717 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:30:57.717 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:31:15.375 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:31:15.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:31:15.528 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T12:31:15.525+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 301389) UID: 0 2024-02-18T12:31:15.529 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T12:31:15.525+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 301389) UID: 0 2024-02-18T12:32:09.462 INFO:teuthology.orchestra.run.smithi116.stdout:generic/310 72s 2024-02-18T12:32:09.467 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/310 2024-02-18T12:32:09.467 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:09.470 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:09.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:09.634 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-02-18T12:32:09.932 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:09.936 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:09.936 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:09.937 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:09.937 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:10.331 INFO:teuthology.orchestra.run.smithi116.stdout:generic/311 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:32:10.334 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/311 2024-02-18T12:32:10.334 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/311 2024-02-18T12:32:10.334 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:10.336 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:10.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:10.353 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-02-18T12:32:10.525 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:10.528 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:10.529 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:10.530 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:10.530 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:10.753 INFO:teuthology.orchestra.run.smithi116.stdout:generic/312 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:32:10.757 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/312 2024-02-18T12:32:10.757 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/312 2024-02-18T12:32:10.757 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:10.759 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:10.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:10.773 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-02-18T12:32:10.946 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:10.950 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:10.950 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:10.951 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:10.951 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:15.223 INFO:teuthology.orchestra.run.smithi116.stdout:generic/313 4s 2024-02-18T12:32:15.228 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/313 2024-02-18T12:32:15.228 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:15.230 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:15.246 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:15.247 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-02-18T12:32:15.421 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:15.425 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:15.425 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:15.426 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:15.426 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:16.075 INFO:teuthology.orchestra.run.smithi116.stdout:generic/314 1s 2024-02-18T12:32:16.080 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/314 2024-02-18T12:32:16.080 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:16.083 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:16.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:16.099 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-02-18T12:32:16.283 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:16.287 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:16.288 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:16.289 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:16.289 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:16.500 INFO:teuthology.orchestra.run.smithi116.stdout:generic/315 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T12:32:16.504 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/315 2024-02-18T12:32:16.504 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/315 2024-02-18T12:32:16.504 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:16.506 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:16.520 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:16.520 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-02-18T12:32:16.703 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:16.707 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:16.707 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:16.708 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:16.708 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:17.026 INFO:teuthology.orchestra.run.smithi116.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T12:32:17.030 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/316 2024-02-18T12:32:17.030 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/316 2024-02-18T12:32:17.030 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:17.032 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:17.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:17.069 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-02-18T12:32:17.481 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:17.486 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:17.487 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:17.488 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:17.488 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:22.185 INFO:teuthology.orchestra.run.smithi116.stdout:generic/317 5s 2024-02-18T12:32:22.190 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/317 2024-02-18T12:32:22.190 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:22.192 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:22.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:22.201 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-02-18T12:32:22.377 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:22.381 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:22.381 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:22.382 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:22.382 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:22.929 INFO:teuthology.orchestra.run.smithi116.stdout:generic/318 0s 2024-02-18T12:32:22.935 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/318 2024-02-18T12:32:22.935 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:22.937 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:22.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:22.946 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-02-18T12:32:23.119 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:23.123 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:23.123 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:23.124 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:23.124 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:23.445 INFO:teuthology.orchestra.run.smithi116.stdout:generic/319 0s 2024-02-18T12:32:23.450 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/319 2024-02-18T12:32:23.451 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:23.453 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:23.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:23.461 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-02-18T12:32:23.643 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:23.646 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:23.647 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:23.648 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:23.648 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:23.863 INFO:teuthology.orchestra.run.smithi116.stdout:generic/320 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:32:23.867 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/320 2024-02-18T12:32:23.867 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/320 2024-02-18T12:32:23.867 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:23.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:23.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:23.881 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-02-18T12:32:24.055 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:24.058 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:24.059 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:24.060 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:24.286 INFO:teuthology.orchestra.run.smithi116.stdout:generic/321 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:32:24.290 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/321 2024-02-18T12:32:24.290 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/321 2024-02-18T12:32:24.290 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:24.293 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:24.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:24.305 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-02-18T12:32:24.480 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:24.483 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:24.484 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:24.485 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:24.485 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:24.687 INFO:teuthology.orchestra.run.smithi116.stdout:generic/322 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:32:24.691 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/322 2024-02-18T12:32:24.691 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/322 2024-02-18T12:32:24.691 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:32:24.693 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:32:24.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:32:24.705 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-02-18T12:32:24.892 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:32:24.895 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:32:24.896 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:32:24.897 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:32:24.897 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:33:28.953 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T12:33:28.950+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 307681) UID: 0 2024-02-18T12:33:28.954 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T12:33:28.950+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 307681) UID: 0 2024-02-18T12:33:28.955 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T12:33:28.950+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 307681) UID: 0 2024-02-18T12:33:28.955 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T12:33:28.950+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 307681) UID: 0 2024-02-18T12:34:26.416 INFO:teuthology.orchestra.run.smithi116.stdout:generic/323 122s 2024-02-18T12:34:26.421 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/323 2024-02-18T12:34:26.421 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:26.423 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:26.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:26.458 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-02-18T12:34:26.865 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:26.869 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:26.870 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:26.871 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:26.871 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:27.138 INFO:teuthology.orchestra.run.smithi116.stdout:generic/324 [not run] defragmentation not supported for fstype "ceph" 2024-02-18T12:34:27.142 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/324 2024-02-18T12:34:27.142 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/324 2024-02-18T12:34:27.142 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:27.144 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:27.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:27.157 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-02-18T12:34:27.334 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:27.338 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:27.338 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:27.339 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:27.339 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:27.582 INFO:teuthology.orchestra.run.smithi116.stdout:generic/325 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:34:27.586 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/325 2024-02-18T12:34:27.586 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/325 2024-02-18T12:34:27.586 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:27.588 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:27.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:27.600 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-02-18T12:34:27.772 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:27.776 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:27.777 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:27.777 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:27.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:28.123 INFO:teuthology.orchestra.run.smithi116.stdout:generic/326 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:28.130 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/326 2024-02-18T12:34:28.131 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/326 2024-02-18T12:34:28.131 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:28.133 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:28.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:28.245 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-02-18T12:34:28.555 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:28.558 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:28.559 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:28.560 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:28.930 INFO:teuthology.orchestra.run.smithi116.stdout:generic/327 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:28.936 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/327 2024-02-18T12:34:28.936 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/327 2024-02-18T12:34:28.936 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:28.940 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:29.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:29.065 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-02-18T12:34:29.372 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:29.376 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:29.376 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:29.377 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:29.377 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:29.727 INFO:teuthology.orchestra.run.smithi116.stdout:generic/328 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:29.732 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/328 2024-02-18T12:34:29.732 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/328 2024-02-18T12:34:29.732 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:29.736 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:29.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:29.847 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-02-18T12:34:30.141 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:30.148 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:30.148 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:30.148 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:30.148 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:30.968 INFO:teuthology.orchestra.run.smithi116.stdout:generic/329 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:30.974 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/329 2024-02-18T12:34:30.974 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/329 2024-02-18T12:34:30.974 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:30.977 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:31.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:31.027 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-02-18T12:34:31.278 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:31.283 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:31.283 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:31.283 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:31.284 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:31.623 INFO:teuthology.orchestra.run.smithi116.stdout:generic/330 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:31.632 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/330 2024-02-18T12:34:31.632 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/330 2024-02-18T12:34:31.632 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:31.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:31.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:31.757 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-02-18T12:34:32.045 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:32.049 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:32.049 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:32.050 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:32.050 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:32.445 INFO:teuthology.orchestra.run.smithi116.stdout:generic/331 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:32.452 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/331 2024-02-18T12:34:32.452 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/331 2024-02-18T12:34:32.452 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:32.454 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:32.537 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:32.538 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-02-18T12:34:32.829 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:32.833 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:32.833 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:32.834 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:32.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:33.271 INFO:teuthology.orchestra.run.smithi116.stdout:generic/332 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:33.279 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/332 2024-02-18T12:34:33.280 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/332 2024-02-18T12:34:33.280 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:33.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:33.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:33.432 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-02-18T12:34:33.688 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:33.692 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:33.692 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:33.693 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:33.693 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:34.069 INFO:teuthology.orchestra.run.smithi116.stdout:generic/333 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:34.076 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/333 2024-02-18T12:34:34.076 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/333 2024-02-18T12:34:34.077 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:34.082 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:34.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:34.280 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-02-18T12:34:34.520 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:34.523 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:34.524 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:34.525 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:34.525 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:34.902 INFO:teuthology.orchestra.run.smithi116.stdout:generic/334 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:34:34.909 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/334 2024-02-18T12:34:34.909 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/334 2024-02-18T12:34:34.909 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:34.911 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:34.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:34.994 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-02-18T12:34:35.288 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:35.291 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:35.291 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:35.293 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:35.293 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:35.687 INFO:teuthology.orchestra.run.smithi116.stdout:generic/335 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:34:35.691 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/335 2024-02-18T12:34:35.691 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/335 2024-02-18T12:34:35.691 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:35.693 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:35.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:35.716 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-02-18T12:34:35.883 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:35.887 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:35.887 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:35.889 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:35.889 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:36.134 INFO:teuthology.orchestra.run.smithi116.stdout:generic/336 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:34:36.138 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/336 2024-02-18T12:34:36.138 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/336 2024-02-18T12:34:36.138 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:36.140 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:36.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:36.155 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-02-18T12:34:36.337 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:36.340 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:36.341 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:36.342 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:36.342 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:37.327 INFO:teuthology.orchestra.run.smithi116.stdout:generic/337 1s 2024-02-18T12:34:37.390 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/337 2024-02-18T12:34:37.390 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:37.392 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:37.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:37.401 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-02-18T12:34:37.569 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:37.573 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:37.573 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:37.574 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:37.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:37.776 INFO:teuthology.orchestra.run.smithi116.stdout:generic/338 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:34:37.780 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/338 2024-02-18T12:34:37.780 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/338 2024-02-18T12:34:37.780 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:34:37.782 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:34:37.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:34:37.814 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-02-18T12:34:37.987 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:34:37.991 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:34:37.991 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:34:37.992 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:34:37.992 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:26.784 INFO:teuthology.orchestra.run.smithi116.stdout:generic/339 48s 2024-02-18T12:35:26.821 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/339 2024-02-18T12:35:26.822 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:26.825 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:26.980 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:26.980 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-02-18T12:35:27.498 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:27.502 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:27.502 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:27.503 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:27.503 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:30.553 INFO:teuthology.orchestra.run.smithi116.stdout:generic/340 3s 2024-02-18T12:35:30.587 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/340 2024-02-18T12:35:30.587 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:30.592 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:30.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:30.629 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-02-18T12:35:30.915 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:30.923 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:30.923 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:30.925 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:30.925 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:31.922 INFO:teuthology.orchestra.run.smithi116.stdout:generic/341 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:35:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/341 2024-02-18T12:35:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/341 2024-02-18T12:35:31.927 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:31.930 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:31.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:31.945 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-02-18T12:35:32.166 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:32.169 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:32.170 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:32.171 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:32.171 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:32.393 INFO:teuthology.orchestra.run.smithi116.stdout:generic/342 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:35:32.396 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/342 2024-02-18T12:35:32.396 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/342 2024-02-18T12:35:32.396 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:32.399 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:32.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:32.411 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-02-18T12:35:32.588 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:32.591 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:32.592 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:32.593 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:32.593 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:33.002 INFO:teuthology.orchestra.run.smithi116.stdout:generic/343 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:35:33.006 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/343 2024-02-18T12:35:33.006 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/343 2024-02-18T12:35:33.006 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:33.008 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:33.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:33.026 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-02-18T12:35:33.202 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:33.206 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:33.207 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:33.208 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:33.208 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:37.600 INFO:teuthology.orchestra.run.smithi116.stdout:generic/344 4s 2024-02-18T12:35:37.609 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/344 2024-02-18T12:35:37.609 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:37.613 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:37.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:37.637 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-02-18T12:35:38.013 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:38.018 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:38.019 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:38.023 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:38.023 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:43.033 INFO:teuthology.orchestra.run.smithi116.stdout:generic/345 4s 2024-02-18T12:35:43.044 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/345 2024-02-18T12:35:43.044 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:43.048 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:43.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:43.060 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-02-18T12:35:43.422 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:43.426 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:43.427 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:43.428 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:43.428 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:46.011 INFO:teuthology.orchestra.run.smithi116.stdout:generic/346 3s 2024-02-18T12:35:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/346 2024-02-18T12:35:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:46.029 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:46.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:46.045 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-02-18T12:35:46.407 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:46.410 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:46.411 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:46.412 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:46.412 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:46.680 INFO:teuthology.orchestra.run.smithi116.stdout:generic/347 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:35:46.684 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/347 2024-02-18T12:35:46.684 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/347 2024-02-18T12:35:46.684 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:46.686 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:46.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:46.700 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-02-18T12:35:46.919 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:46.924 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:46.925 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:46.926 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:46.926 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:47.317 INFO:teuthology.orchestra.run.smithi116.stdout:generic/348 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:35:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/348 2024-02-18T12:35:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/348 2024-02-18T12:35:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:47.328 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:47.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:47.369 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-02-18T12:35:47.554 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:47.558 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:47.558 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:47.559 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:47.559 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:47.900 INFO:teuthology.orchestra.run.smithi116.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-02-18T12:35:47.903 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/349 2024-02-18T12:35:47.904 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/349 2024-02-18T12:35:47.904 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:47.906 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:47.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:47.924 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-02-18T12:35:48.102 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:48.106 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:48.106 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:48.108 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:48.108 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:49.244 INFO:teuthology.orchestra.run.smithi116.stdout:generic/350 1s 2024-02-18T12:35:49.250 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/350 2024-02-18T12:35:49.250 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:49.252 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:49.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:49.264 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-02-18T12:35:49.447 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:49.451 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:49.451 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:49.452 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:49.452 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:49.679 INFO:teuthology.orchestra.run.smithi116.stdout:generic/351 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:35:49.683 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/351 2024-02-18T12:35:49.683 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/351 2024-02-18T12:35:49.683 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:49.686 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:49.702 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:49.702 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-02-18T12:35:49.880 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:49.884 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:49.884 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:49.885 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:49.886 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:50.246 INFO:teuthology.orchestra.run.smithi116.stdout:generic/352 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:35:50.254 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/352 2024-02-18T12:35:50.254 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/352 2024-02-18T12:35:50.254 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:50.256 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:50.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:50.565 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-02-18T12:35:50.757 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:50.760 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:50.761 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:50.762 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:50.762 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:51.125 INFO:teuthology.orchestra.run.smithi116.stdout:generic/353 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:35:51.133 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/353 2024-02-18T12:35:51.133 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/353 2024-02-18T12:35:51.134 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:51.136 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:51.419 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:51.419 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-02-18T12:35:51.601 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:51.604 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:51.605 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:51.606 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:51.606 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:55.676 INFO:teuthology.orchestra.run.smithi116.stdout:generic/354 4s 2024-02-18T12:35:55.684 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/354 2024-02-18T12:35:55.685 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:55.687 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:55.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:55.699 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-02-18T12:35:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:56.096 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:56.096 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:56.098 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:56.098 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:57.298 INFO:teuthology.orchestra.run.smithi116.stdout:generic/355 1s 2024-02-18T12:35:57.304 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/355 2024-02-18T12:35:57.304 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:57.306 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:57.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:57.362 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-02-18T12:35:57.741 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:57.745 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:57.746 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:57.747 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:57.747 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:58.104 INFO:teuthology.orchestra.run.smithi116.stdout:generic/356 [not run] swapfiles are not supported 2024-02-18T12:35:58.109 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/356 2024-02-18T12:35:58.109 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/356 2024-02-18T12:35:58.109 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:58.111 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:58.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:58.128 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-02-18T12:35:58.312 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:58.316 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:58.316 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:58.317 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:58.317 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:58.639 INFO:teuthology.orchestra.run.smithi116.stdout:generic/357 [not run] swapfiles are not supported 2024-02-18T12:35:58.643 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/357 2024-02-18T12:35:58.644 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/357 2024-02-18T12:35:58.644 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:58.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:58.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:58.663 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-02-18T12:35:58.851 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:58.855 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:58.855 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:58.856 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:58.856 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:59.208 INFO:teuthology.orchestra.run.smithi116.stdout:generic/358 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:35:59.218 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/358 2024-02-18T12:35:59.218 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/358 2024-02-18T12:35:59.218 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:35:59.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:35:59.527 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:35:59.527 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-02-18T12:35:59.715 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:35:59.719 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:35:59.719 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:35:59.720 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:35:59.720 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:00.085 INFO:teuthology.orchestra.run.smithi116.stdout:generic/359 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:36:00.095 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/359 2024-02-18T12:36:00.095 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/359 2024-02-18T12:36:00.095 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:00.100 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:00.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:00.383 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-02-18T12:36:00.577 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:00.580 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:00.581 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:00.581 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:00.582 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:00.850 INFO:teuthology.orchestra.run.smithi116.stdout:generic/360 0s 2024-02-18T12:36:00.856 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/360 2024-02-18T12:36:00.856 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:00.858 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:00.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:00.877 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-02-18T12:36:01.049 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:01.052 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:01.053 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:01.054 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:01.054 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:01.302 INFO:teuthology.orchestra.run.smithi116.stdout:generic/361 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:36:01.306 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/361 2024-02-18T12:36:01.306 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/361 2024-02-18T12:36:01.306 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:01.308 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:01.322 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:01.322 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-02-18T12:36:01.504 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:01.507 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:01.508 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:01.509 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:01.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:01.708 INFO:teuthology.orchestra.run.smithi116.stdout:generic/362 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:01.712 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/362 2024-02-18T12:36:01.712 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/362 2024-02-18T12:36:01.712 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:01.714 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:01.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:01.728 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-02-18T12:36:01.904 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:01.908 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:01.908 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:01.909 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:01.909 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:02.108 INFO:teuthology.orchestra.run.smithi116.stdout:generic/363 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:02.112 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/363 2024-02-18T12:36:02.112 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/363 2024-02-18T12:36:02.112 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:02.114 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:02.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:02.127 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-02-18T12:36:02.315 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:02.319 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:02.320 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:02.321 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:02.321 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:02.556 INFO:teuthology.orchestra.run.smithi116.stdout:generic/364 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:02.560 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/364 2024-02-18T12:36:02.560 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/364 2024-02-18T12:36:02.560 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:02.562 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:02.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:02.575 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-02-18T12:36:02.759 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:02.763 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:02.763 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:02.764 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:02.764 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:02.971 INFO:teuthology.orchestra.run.smithi116.stdout:generic/365 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:02.975 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/365 2024-02-18T12:36:02.975 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/365 2024-02-18T12:36:02.975 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:02.977 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:02.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:02.991 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-02-18T12:36:03.165 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:03.169 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:03.169 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:03.170 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:03.170 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:03.369 INFO:teuthology.orchestra.run.smithi116.stdout:generic/366 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:03.373 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/366 2024-02-18T12:36:03.373 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/366 2024-02-18T12:36:03.373 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:03.376 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:03.402 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:03.402 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-02-18T12:36:03.578 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:03.582 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:03.582 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:03.583 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:03.805 INFO:teuthology.orchestra.run.smithi116.stdout:generic/367 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:03.809 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/367 2024-02-18T12:36:03.809 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/367 2024-02-18T12:36:03.809 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:03.812 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:03.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:03.832 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-02-18T12:36:04.008 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:04.012 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:04.012 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:04.013 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:04.014 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:04.215 INFO:teuthology.orchestra.run.smithi116.stdout:generic/368 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:04.219 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/368 2024-02-18T12:36:04.219 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/368 2024-02-18T12:36:04.219 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:04.221 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:04.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:04.234 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-02-18T12:36:04.413 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:04.417 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:04.417 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:04.418 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:04.418 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:04.629 INFO:teuthology.orchestra.run.smithi116.stdout:generic/369 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:04.633 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/369 2024-02-18T12:36:04.633 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/369 2024-02-18T12:36:04.634 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:04.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:04.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:04.650 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-02-18T12:36:04.841 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:04.847 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:04.848 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:04.848 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:04.848 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:05.051 INFO:teuthology.orchestra.run.smithi116.stdout:generic/370 [not run] this test requires richacl support on $SCRATCH_DEV 2024-02-18T12:36:05.053 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/370 2024-02-18T12:36:05.054 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/370 2024-02-18T12:36:05.054 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:05.056 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:05.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:05.070 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-02-18T12:36:05.250 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:05.256 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:05.256 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:05.256 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:05.256 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:05.547 INFO:teuthology.orchestra.run.smithi116.stdout:generic/371 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:36:05.551 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/371 2024-02-18T12:36:05.551 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/371 2024-02-18T12:36:05.551 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:05.553 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:05.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:05.570 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-02-18T12:36:05.743 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:05.747 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:05.747 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:05.748 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:05.748 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:06.145 INFO:teuthology.orchestra.run.smithi116.stdout:generic/372 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:36:06.153 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/372 2024-02-18T12:36:06.153 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/372 2024-02-18T12:36:06.153 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:06.157 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:06.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:06.415 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-02-18T12:36:06.600 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:06.604 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:06.604 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:06.605 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:06.605 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:07.111 INFO:teuthology.orchestra.run.smithi116.stdout:generic/373 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:36:07.118 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/373 2024-02-18T12:36:07.118 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/373 2024-02-18T12:36:07.118 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:07.122 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:07.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:07.376 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-02-18T12:36:07.559 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:07.562 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:07.563 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:07.564 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:07.564 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:07.950 INFO:teuthology.orchestra.run.smithi116.stdout:generic/374 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-02-18T12:36:08.018 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/374 2024-02-18T12:36:08.018 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/374 2024-02-18T12:36:08.018 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:08.021 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:08.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:08.356 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-02-18T12:36:08.534 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:08.538 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:08.538 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:08.539 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:08.540 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:08.837 INFO:teuthology.orchestra.run.smithi116.stdout:generic/375 0s 2024-02-18T12:36:08.842 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/375 2024-02-18T12:36:08.842 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:08.845 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:08.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:08.901 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-02-18T12:36:09.314 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:09.318 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:09.319 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:09.320 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:09.320 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:09.533 INFO:teuthology.orchestra.run.smithi116.stdout:generic/376 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:36:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/376 2024-02-18T12:36:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/376 2024-02-18T12:36:09.537 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:09.539 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:09.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:09.553 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-02-18T12:36:09.730 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:09.734 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:09.734 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:09.735 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:09.735 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:10.130 INFO:teuthology.orchestra.run.smithi116.stdout:generic/377 1s 2024-02-18T12:36:10.135 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/377 2024-02-18T12:36:10.135 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:10.137 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:10.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:10.146 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-02-18T12:36:10.325 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:10.329 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:10.330 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:10.330 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:10.330 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:10.785 INFO:teuthology.orchestra.run.smithi116.stdout:generic/378 0s 2024-02-18T12:36:10.787 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/378 2024-02-18T12:36:10.788 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:10.790 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:10.848 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:10.848 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-02-18T12:36:11.246 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:11.250 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:11.250 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:11.251 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:11.251 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:11.502 INFO:teuthology.orchestra.run.smithi116.stdout:generic/379 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:11.506 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/379 2024-02-18T12:36:11.506 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/379 2024-02-18T12:36:11.506 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:11.508 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:11.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:11.530 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-02-18T12:36:11.715 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:11.719 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:11.720 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:11.720 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:11.720 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:11.924 INFO:teuthology.orchestra.run.smithi116.stdout:generic/380 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:11.928 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/380 2024-02-18T12:36:11.928 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/380 2024-02-18T12:36:11.928 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:11.930 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:11.943 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:11.943 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-02-18T12:36:12.121 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:12.125 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:12.126 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:12.127 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:12.127 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:12.332 INFO:teuthology.orchestra.run.smithi116.stdout:generic/381 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:12.336 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/381 2024-02-18T12:36:12.336 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/381 2024-02-18T12:36:12.336 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:12.339 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:12.352 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:12.352 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-02-18T12:36:12.526 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:12.530 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:12.531 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:12.532 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:12.532 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:12.743 INFO:teuthology.orchestra.run.smithi116.stdout:generic/382 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/382 2024-02-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/382 2024-02-18T12:36:12.747 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:12.749 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:12.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:12.762 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-02-18T12:36:12.942 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:12.946 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:12.946 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:12.947 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:12.947 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:13.166 INFO:teuthology.orchestra.run.smithi116.stdout:generic/383 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:13.170 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/383 2024-02-18T12:36:13.170 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/383 2024-02-18T12:36:13.170 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:13.172 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:13.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:13.186 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-02-18T12:36:13.364 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:13.368 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:13.368 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:13.369 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:13.369 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:13.579 INFO:teuthology.orchestra.run.smithi116.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:13.583 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/384 2024-02-18T12:36:13.583 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/384 2024-02-18T12:36:13.583 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:13.585 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:13.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:13.598 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-02-18T12:36:13.771 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:13.775 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:13.776 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:13.776 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:13.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:13.983 INFO:teuthology.orchestra.run.smithi116.stdout:generic/385 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:13.987 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/385 2024-02-18T12:36:13.987 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/385 2024-02-18T12:36:13.988 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:13.990 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:14.003 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:14.004 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-02-18T12:36:14.181 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:14.185 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:14.185 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:14.186 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:14.187 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:14.387 INFO:teuthology.orchestra.run.smithi116.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:14.391 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/386 2024-02-18T12:36:14.391 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/386 2024-02-18T12:36:14.391 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:14.394 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:14.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:14.407 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-02-18T12:36:14.584 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:14.588 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:14.589 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:14.590 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:14.590 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:14.935 INFO:teuthology.orchestra.run.smithi116.stdout:generic/387 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/387 2024-02-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/387 2024-02-18T12:36:14.941 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:14.944 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:15.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:15.186 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-02-18T12:36:15.406 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:15.409 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:15.410 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:15.411 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:15.411 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:15.625 INFO:teuthology.orchestra.run.smithi116.stdout:generic/388 [not run] require 172.21.15.116:6789:/scratch to be local device 2024-02-18T12:36:15.629 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/388 2024-02-18T12:36:15.629 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/388 2024-02-18T12:36:15.629 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:15.631 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:15.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:15.648 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-02-18T12:36:15.831 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:15.835 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:15.836 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:15.837 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:15.837 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:16.051 INFO:teuthology.orchestra.run.smithi116.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-02-18T12:36:16.055 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/389 2024-02-18T12:36:16.055 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/389 2024-02-18T12:36:16.055 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:16.058 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:16.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:16.071 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-02-18T12:36:16.245 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:16.248 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:16.249 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:16.250 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:16.250 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:16.495 INFO:teuthology.orchestra.run.smithi116.stdout:generic/390 [not run] ceph does not support freezing 2024-02-18T12:36:16.499 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/390 2024-02-18T12:36:16.499 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/390 2024-02-18T12:36:16.500 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:16.502 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:16.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:16.536 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-02-18T12:36:16.713 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:16.717 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:16.718 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:16.719 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:16.719 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:16.954 INFO:teuthology.orchestra.run.smithi116.stdout:generic/391 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:36:16.958 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/391 2024-02-18T12:36:16.958 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/391 2024-02-18T12:36:16.958 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:16.960 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:16.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:16.977 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-02-18T12:36:17.146 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:17.150 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:17.150 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:17.151 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:17.151 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:17.454 INFO:teuthology.orchestra.run.smithi116.stdout:generic/392 [not run] ceph does not support shutdown 2024-02-18T12:36:17.463 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/392 2024-02-18T12:36:17.463 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/392 2024-02-18T12:36:17.463 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:17.465 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:17.478 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:17.478 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-02-18T12:36:17.655 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:17.658 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:17.658 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:17.659 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:17.659 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:18.306 INFO:teuthology.orchestra.run.smithi116.stdout:generic/393 1s 2024-02-18T12:36:18.311 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/393 2024-02-18T12:36:18.311 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:18.313 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:18.322 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:18.322 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-02-18T12:36:18.504 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:18.507 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:18.508 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:18.509 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:18.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:18.747 INFO:teuthology.orchestra.run.smithi116.stdout:generic/394 0s 2024-02-18T12:36:18.752 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/394 2024-02-18T12:36:18.753 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:18.755 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:18.804 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:18.804 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-02-18T12:36:19.244 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:19.248 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:19.249 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:19.250 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:19.250 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:19.736 INFO:teuthology.orchestra.run.smithi116.stdout:generic/395 [not run] kernel does not support ceph encryption 2024-02-18T12:36:19.759 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/395 2024-02-18T12:36:19.759 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/395 2024-02-18T12:36:19.759 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:19.761 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:19.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:19.778 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-02-18T12:36:20.048 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:20.053 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:20.054 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:20.056 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:20.056 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:20.499 INFO:teuthology.orchestra.run.smithi116.stdout:generic/396 [not run] kernel does not support ceph encryption 2024-02-18T12:36:20.507 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/396 2024-02-18T12:36:20.508 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/396 2024-02-18T12:36:20.508 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:20.510 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:20.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:20.524 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-02-18T12:36:20.714 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:20.718 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:20.718 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:20.719 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:20.719 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:21.083 INFO:teuthology.orchestra.run.smithi116.stdout:generic/397 [not run] kernel does not support ceph encryption 2024-02-18T12:36:21.091 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/397 2024-02-18T12:36:21.092 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/397 2024-02-18T12:36:21.092 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:21.094 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:21.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:21.117 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-02-18T12:36:21.290 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:21.294 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:21.294 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:21.295 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:21.296 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:21.647 INFO:teuthology.orchestra.run.smithi116.stdout:generic/398 [not run] kernel does not support ceph encryption 2024-02-18T12:36:21.655 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/398 2024-02-18T12:36:21.656 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/398 2024-02-18T12:36:21.656 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:21.658 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:21.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:21.671 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-02-18T12:36:21.856 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:21.860 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:21.860 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:21.861 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:21.861 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:22.182 INFO:teuthology.orchestra.run.smithi116.stdout:generic/399 [not run] kernel does not support ceph encryption 2024-02-18T12:36:22.191 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/399 2024-02-18T12:36:22.191 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/399 2024-02-18T12:36:22.191 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:22.193 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:22.206 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:22.206 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-02-18T12:36:22.386 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:22.390 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:22.391 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:22.392 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:22.392 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:22.600 INFO:teuthology.orchestra.run.smithi116.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T12:36:22.604 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/400 2024-02-18T12:36:22.604 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/400 2024-02-18T12:36:22.604 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:22.606 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:22.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:22.620 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-02-18T12:36:22.799 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:22.803 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:22.803 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:22.804 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:22.805 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:23.265 INFO:teuthology.orchestra.run.smithi116.stdout:generic/401 1s 2024-02-18T12:36:23.276 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/401 2024-02-18T12:36:23.276 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:23.277 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:23.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:23.289 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-02-18T12:36:23.675 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:23.678 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:23.679 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:23.680 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:23.680 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:23.997 INFO:teuthology.orchestra.run.smithi116.stdout:generic/402 [not run] filesystem ceph timestamp bounds are unknown 2024-02-18T12:36:24.005 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/402 2024-02-18T12:36:24.005 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/402 2024-02-18T12:36:24.005 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:24.007 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:24.024 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:24.024 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-02-18T12:36:24.199 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:24.202 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:24.203 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:24.204 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:24.204 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:24.675 INFO:teuthology.orchestra.run.smithi116.stdout:generic/403 0s 2024-02-18T12:36:24.681 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/403 2024-02-18T12:36:24.681 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:24.683 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:24.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:24.692 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-02-18T12:36:24.868 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:24.872 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:24.872 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:24.873 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:24.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:25.098 INFO:teuthology.orchestra.run.smithi116.stdout:generic/404 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:36:25.102 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/404 2024-02-18T12:36:25.102 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/404 2024-02-18T12:36:25.102 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:25.104 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:25.121 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:25.121 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-02-18T12:36:25.299 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:25.303 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:25.303 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:25.304 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:25.304 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:25.579 INFO:teuthology.orchestra.run.smithi116.stdout:generic/405 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:36:25.582 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/405 2024-02-18T12:36:25.582 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/405 2024-02-18T12:36:25.582 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:25.585 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:25.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:25.598 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-02-18T12:36:25.774 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:25.778 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:25.778 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:25.780 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:25.780 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:27.408 INFO:teuthology.orchestra.run.smithi116.stdout:generic/406 2s 2024-02-18T12:36:27.413 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/406 2024-02-18T12:36:27.413 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:27.415 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:27.423 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:27.424 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-02-18T12:36:27.603 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:27.607 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:27.607 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:27.608 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:27.608 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:27.877 INFO:teuthology.orchestra.run.smithi116.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:36:27.883 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/407 2024-02-18T12:36:27.883 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/407 2024-02-18T12:36:27.883 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:27.885 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:27.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:27.987 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-02-18T12:36:28.469 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:28.473 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:28.474 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:28.475 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:28.475 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:28.731 INFO:teuthology.orchestra.run.smithi116.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T12:36:28.734 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/408 2024-02-18T12:36:28.734 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/408 2024-02-18T12:36:28.734 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:28.736 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:28.802 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:28.802 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-02-18T12:36:29.219 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:29.222 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:29.223 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:29.224 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:29.224 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:29.437 INFO:teuthology.orchestra.run.smithi116.stdout:generic/409 [not run] require 172.21.15.116:6789:/scratch to be local device 2024-02-18T12:36:29.441 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/409 2024-02-18T12:36:29.441 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/409 2024-02-18T12:36:29.441 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:29.443 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:29.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:29.457 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-02-18T12:36:29.629 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:29.632 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:29.633 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:29.634 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:29.634 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:29.856 INFO:teuthology.orchestra.run.smithi116.stdout:generic/410 [not run] require 172.21.15.116:6789:/scratch to be local device 2024-02-18T12:36:29.860 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/410 2024-02-18T12:36:29.861 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/410 2024-02-18T12:36:29.861 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:29.863 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:29.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:29.877 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-02-18T12:36:30.059 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:30.063 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:30.064 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:30.064 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:30.065 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:30.277 INFO:teuthology.orchestra.run.smithi116.stdout:generic/411 [not run] require 172.21.15.116:6789:/scratch to be local device 2024-02-18T12:36:30.281 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/411 2024-02-18T12:36:30.281 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/411 2024-02-18T12:36:30.281 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:30.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:30.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:30.296 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-02-18T12:36:30.471 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:30.474 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:30.475 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:30.476 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:30.476 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:30.950 INFO:teuthology.orchestra.run.smithi116.stdout:generic/412 0s 2024-02-18T12:36:30.994 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/412 2024-02-18T12:36:30.994 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:30.994 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:30.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:30.995 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-02-18T12:36:31.184 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:31.188 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:31.188 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:31.189 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:31.189 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:31.464 INFO:teuthology.orchestra.run.smithi116.stdout:generic/413 [not run] mount 172.21.15.116:6789:/scratch with dax failed 2024-02-18T12:36:31.468 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/413 2024-02-18T12:36:31.468 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/413 2024-02-18T12:36:31.468 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:31.471 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:31.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:31.492 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-02-18T12:36:31.669 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:31.673 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:31.674 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:31.675 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:31.675 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:32.078 INFO:teuthology.orchestra.run.smithi116.stdout:generic/414 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:36:32.084 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/414 2024-02-18T12:36:32.084 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/414 2024-02-18T12:36:32.084 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:32.087 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:32.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:32.417 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-02-18T12:36:32.601 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:32.605 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:32.605 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:32.606 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:32.607 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:32.953 INFO:teuthology.orchestra.run.smithi116.stdout:generic/415 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:36:32.958 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/415 2024-02-18T12:36:32.958 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/415 2024-02-18T12:36:32.958 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:32.967 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:33.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:33.212 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-02-18T12:36:33.527 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:33.530 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:33.531 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:33.532 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:33.532 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:33.847 INFO:teuthology.orchestra.run.smithi116.stdout:generic/416 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:36:33.851 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/416 2024-02-18T12:36:33.851 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/416 2024-02-18T12:36:33.851 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:33.854 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:33.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:33.867 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-02-18T12:36:34.062 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:34.066 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:34.066 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:34.067 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:34.067 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:34.359 INFO:teuthology.orchestra.run.smithi116.stdout:generic/417 [not run] ceph does not support shutdown 2024-02-18T12:36:34.368 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/417 2024-02-18T12:36:34.368 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/417 2024-02-18T12:36:34.368 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:34.370 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:34.384 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:34.384 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-02-18T12:36:34.565 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:34.568 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:34.569 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:34.570 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:34.570 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:34.822 INFO:teuthology.orchestra.run.smithi116.stdout:generic/418 [not run] require 172.21.15.116:6789:/test to be valid block disk 2024-02-18T12:36:34.826 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/418 2024-02-18T12:36:34.826 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/418 2024-02-18T12:36:34.826 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:34.828 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:34.846 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:34.847 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-02-18T12:36:35.031 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:35.035 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:35.035 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:35.036 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:35.037 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:35.382 INFO:teuthology.orchestra.run.smithi116.stdout:generic/419 [not run] kernel does not support ceph encryption 2024-02-18T12:36:35.391 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/419 2024-02-18T12:36:35.391 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/419 2024-02-18T12:36:35.391 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:35.393 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:35.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:35.407 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-02-18T12:36:35.693 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:35.708 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:35.709 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:35.709 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:35.709 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:36.313 INFO:teuthology.orchestra.run.smithi116.stdout:generic/420 1s 2024-02-18T12:36:36.330 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/420 2024-02-18T12:36:36.330 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:36.332 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:36.384 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:36.384 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-02-18T12:36:36.791 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:36.796 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:36.796 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:36.797 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:36.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:37.117 INFO:teuthology.orchestra.run.smithi116.stdout:generic/421 [not run] kernel does not support ceph encryption 2024-02-18T12:36:37.136 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/421 2024-02-18T12:36:37.136 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/421 2024-02-18T12:36:37.136 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:37.138 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:37.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:37.151 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-02-18T12:36:37.330 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:37.334 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:37.335 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:37.336 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:37.336 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:37.572 INFO:teuthology.orchestra.run.smithi116.stdout:generic/422 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T12:36:37.576 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/422 2024-02-18T12:36:37.576 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/422 2024-02-18T12:36:37.576 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:37.579 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:37.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:37.598 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-02-18T12:36:37.783 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:37.787 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:37.787 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:37.788 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:37.788 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:38.172 INFO:teuthology.orchestra.run.smithi116.stdout:generic/423 1s 2024-02-18T12:36:38.177 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/423 2024-02-18T12:36:38.177 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:38.179 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:38.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:38.241 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-02-18T12:36:38.648 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:38.652 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:38.652 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:38.653 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:38.653 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:38.975 INFO:teuthology.orchestra.run.smithi116.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-02-18T12:36:38.979 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/424 2024-02-18T12:36:38.980 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/424 2024-02-18T12:36:38.980 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:38.982 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:39.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:39.001 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-02-18T12:36:39.174 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:39.178 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:39.179 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:39.180 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:39.180 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:39.681 INFO:teuthology.orchestra.run.smithi116.stdout:generic/425 [not run] xfs_io fiemap -a failed (old kernel/wrong fs?) 2024-02-18T12:36:39.685 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/425 2024-02-18T12:36:39.685 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/425 2024-02-18T12:36:39.685 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:36:39.687 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:36:39.733 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:36:39.733 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-02-18T12:36:40.110 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:36:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:36:40.114 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:36:40.115 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:36:40.115 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:09.098 INFO:teuthology.orchestra.run.smithi116.stdout:generic/426 28s 2024-02-18T12:37:09.244 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/426 2024-02-18T12:37:09.245 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:09.247 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:09.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:09.477 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-02-18T12:37:12.259 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:12.263 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:12.264 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:12.265 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:12.265 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:12.802 INFO:teuthology.orchestra.run.smithi116.stdout:generic/427 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:37:12.806 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/427 2024-02-18T12:37:12.806 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/427 2024-02-18T12:37:12.806 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:12.808 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:12.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:12.825 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-02-18T12:37:13.013 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:13.017 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:13.017 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:13.018 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:13.018 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:13.283 INFO:teuthology.orchestra.run.smithi116.stdout:generic/428 0s 2024-02-18T12:37:13.288 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/428 2024-02-18T12:37:13.288 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:13.291 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:13.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:13.309 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-02-18T12:37:13.491 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:13.495 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:13.495 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:13.496 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:13.496 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:13.810 INFO:teuthology.orchestra.run.smithi116.stdout:generic/429 [not run] kernel does not support ceph encryption 2024-02-18T12:37:13.818 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/429 2024-02-18T12:37:13.818 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/429 2024-02-18T12:37:13.818 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:13.820 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:13.832 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:13.832 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-02-18T12:37:14.006 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:14.010 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:14.010 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:14.011 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:14.012 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:14.334 INFO:teuthology.orchestra.run.smithi116.stdout:generic/430 0s 2024-02-18T12:37:14.339 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/430 2024-02-18T12:37:14.339 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:14.341 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:14.402 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:14.402 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-02-18T12:37:14.770 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:14.774 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:14.774 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:14.775 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:14.775 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:15.050 INFO:teuthology.orchestra.run.smithi116.stdout:generic/431 1s 2024-02-18T12:37:15.056 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/431 2024-02-18T12:37:15.056 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:15.058 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:15.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:15.148 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-02-18T12:37:15.549 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:15.553 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:15.553 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:15.554 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:15.554 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:15.854 INFO:teuthology.orchestra.run.smithi116.stdout:generic/432 0s 2024-02-18T12:37:15.859 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/432 2024-02-18T12:37:15.859 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:15.861 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:15.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:15.913 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-02-18T12:37:16.313 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:16.317 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:16.317 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:16.318 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:16.318 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:16.595 INFO:teuthology.orchestra.run.smithi116.stdout:generic/433 0s 2024-02-18T12:37:16.600 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/433 2024-02-18T12:37:16.600 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:16.603 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:16.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:16.663 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-02-18T12:37:17.091 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:17.096 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:17.096 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:17.101 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:17.101 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:17.397 INFO:teuthology.orchestra.run.smithi116.stdout:generic/434 0s 2024-02-18T12:37:17.402 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/434 2024-02-18T12:37:17.403 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:17.405 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:17.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:17.470 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-02-18T12:37:17.812 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:17.815 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:17.816 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:17.817 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:17.817 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:18.187 INFO:teuthology.orchestra.run.smithi116.stdout:generic/435 [not run] kernel does not support ceph encryption 2024-02-18T12:37:18.196 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/435 2024-02-18T12:37:18.196 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/435 2024-02-18T12:37:18.196 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:18.198 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:18.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:18.221 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-02-18T12:37:18.404 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:18.408 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:18.408 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:18.409 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:18.409 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:19.019 INFO:teuthology.orchestra.run.smithi116.stdout:generic/436 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:37:19.023 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/436 2024-02-18T12:37:19.023 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/436 2024-02-18T12:37:19.023 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:19.025 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:19.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:19.072 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-02-18T12:37:19.471 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:19.475 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:19.475 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:19.476 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:19.476 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:20.306 INFO:teuthology.orchestra.run.smithi116.stdout:generic/437 1s 2024-02-18T12:37:20.313 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/437 2024-02-18T12:37:20.313 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:37:20.314 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:37:20.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:37:20.333 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-02-18T12:37:20.506 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:37:20.510 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:37:20.510 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:37:20.511 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:37:20.511 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:24.888 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:38:24.891 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:38:24.922 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T12:38:24.921+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50371) UID: 0 2024-02-18T12:38:24.923 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T12:38:24.921+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50371) UID: 0 2024-02-18T12:38:24.923 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T12:38:24.921+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50371) UID: 0 2024-02-18T12:38:24.923 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T12:38:24.921+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50371) UID: 0 2024-02-18T12:38:34.233 INFO:teuthology.orchestra.run.smithi116.stdout:generic/438 74s 2024-02-18T12:38:34.238 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/438 2024-02-18T12:38:34.238 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:34.241 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:34.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:34.258 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-02-18T12:38:34.430 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:34.434 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:34.434 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:34.435 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:34.435 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:35.301 INFO:teuthology.orchestra.run.smithi116.stdout:generic/439 1s 2024-02-18T12:38:35.306 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/439 2024-02-18T12:38:35.306 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:35.309 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:35.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:35.326 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-02-18T12:38:35.507 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:35.511 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:35.511 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:35.512 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:35.512 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:35.828 INFO:teuthology.orchestra.run.smithi116.stdout:generic/440 [not run] kernel does not support ceph encryption 2024-02-18T12:38:35.835 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/440 2024-02-18T12:38:35.836 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/440 2024-02-18T12:38:35.836 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:35.838 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:35.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:35.850 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-02-18T12:38:36.032 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:36.036 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:36.036 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:36.037 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:36.038 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:36.274 INFO:teuthology.orchestra.run.smithi116.stdout:generic/441 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:38:36.278 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/441 2024-02-18T12:38:36.278 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/441 2024-02-18T12:38:36.278 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:36.280 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:36.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:36.292 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-02-18T12:38:36.468 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:36.472 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:36.473 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:36.473 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:36.474 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:36.702 INFO:teuthology.orchestra.run.smithi116.stdout:generic/442 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:38:36.706 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/442 2024-02-18T12:38:36.706 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/442 2024-02-18T12:38:36.706 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:36.709 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:36.721 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:36.721 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-02-18T12:38:36.899 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:36.903 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:36.904 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:36.904 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:37.131 INFO:teuthology.orchestra.run.smithi116.stdout:generic/443 1s 2024-02-18T12:38:37.136 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/443 2024-02-18T12:38:37.136 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:37.139 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:37.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:37.156 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-02-18T12:38:37.335 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:37.339 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:37.339 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:37.340 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:37.340 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:37.619 INFO:teuthology.orchestra.run.smithi116.stdout:generic/444 0s 2024-02-18T12:38:37.624 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/444 2024-02-18T12:38:37.625 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:37.627 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:37.644 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:37.644 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-02-18T12:38:37.824 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:37.828 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:37.828 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:37.831 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:37.832 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:38.378 INFO:teuthology.orchestra.run.smithi116.stdout:generic/445 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:38:38.382 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/445 2024-02-18T12:38:38.382 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/445 2024-02-18T12:38:38.382 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:38.384 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:38.420 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:38.420 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-02-18T12:38:38.832 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:38.836 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:38.837 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:38.838 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:38.838 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:55.706 INFO:teuthology.orchestra.run.smithi116.stdout:generic/446 17s 2024-02-18T12:38:55.715 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/446 2024-02-18T12:38:55.715 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:55.717 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:55.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:55.738 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-02-18T12:38:56.094 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:56.098 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:56.098 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:56.099 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:56.099 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:56.443 INFO:teuthology.orchestra.run.smithi116.stdout:generic/447 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:38:56.450 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/447 2024-02-18T12:38:56.450 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/447 2024-02-18T12:38:56.450 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:56.452 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:56.560 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:56.561 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-02-18T12:38:56.820 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:56.824 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:56.824 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:56.826 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:56.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:57.729 INFO:teuthology.orchestra.run.smithi116.stdout:generic/448 1s 2024-02-18T12:38:57.735 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/448 2024-02-18T12:38:57.735 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:57.738 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:57.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:57.777 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-02-18T12:38:58.180 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:58.184 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:58.185 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:58.186 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:58.186 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:58.444 INFO:teuthology.orchestra.run.smithi116.stdout:generic/449 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T12:38:58.448 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/449 2024-02-18T12:38:58.448 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/449 2024-02-18T12:38:58.448 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:58.450 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:58.501 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:58.502 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-02-18T12:38:58.865 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:58.871 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:58.872 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:58.872 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:58.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:59.411 INFO:teuthology.orchestra.run.smithi116.stdout:generic/450 1s 2024-02-18T12:38:59.416 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/450 2024-02-18T12:38:59.416 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:38:59.418 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:38:59.469 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:38:59.470 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-02-18T12:38:59.830 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:38:59.834 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:38:59.834 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:38:59.835 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:38:59.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:30.259 INFO:teuthology.orchestra.run.smithi116.stdout:generic/451 31s 2024-02-18T12:39:30.264 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/451 2024-02-18T12:39:30.264 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:30.266 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:30.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:30.321 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-02-18T12:39:30.708 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:30.712 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:30.713 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:30.714 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:30.714 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:31.070 INFO:teuthology.orchestra.run.smithi116.stdout:generic/452 1s 2024-02-18T12:39:31.076 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/452 2024-02-18T12:39:31.076 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:31.078 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:31.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:31.087 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-02-18T12:39:31.263 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:31.266 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:31.267 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:31.268 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:31.268 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:31.473 INFO:teuthology.orchestra.run.smithi116.stdout:generic/453 [not run] ceph does not allow unrestricted byte streams for names 2024-02-18T12:39:31.476 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/453 2024-02-18T12:39:31.476 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/453 2024-02-18T12:39:31.477 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:31.479 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:31.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:31.493 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-02-18T12:39:31.667 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:31.671 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:31.671 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:31.672 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:31.672 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:31.901 INFO:teuthology.orchestra.run.smithi116.stdout:generic/454 [not run] ceph does not allow unrestricted byte streams for names 2024-02-18T12:39:31.905 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/454 2024-02-18T12:39:31.905 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/454 2024-02-18T12:39:31.905 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:31.907 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:31.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:31.925 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-02-18T12:39:32.110 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:32.114 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:32.114 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:32.115 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:32.115 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:32.395 INFO:teuthology.orchestra.run.smithi116.stdout:generic/455 [not run] This test requires a valid $LOGWRITES_DEV 2024-02-18T12:39:32.399 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/455 2024-02-18T12:39:32.400 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/455 2024-02-18T12:39:32.400 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:32.402 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:32.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:32.416 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-02-18T12:39:32.600 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:32.604 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:32.604 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:32.605 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:32.605 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:32.833 INFO:teuthology.orchestra.run.smithi116.stdout:generic/456 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T12:39:32.837 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/456 2024-02-18T12:39:32.837 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/456 2024-02-18T12:39:32.837 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:32.839 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:32.855 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:32.855 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-02-18T12:39:33.034 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:33.038 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:33.038 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:33.039 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:33.039 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:33.467 INFO:teuthology.orchestra.run.smithi116.stdout:generic/457 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:39:33.474 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/457 2024-02-18T12:39:33.474 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/457 2024-02-18T12:39:33.474 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:33.477 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:33.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:33.601 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-02-18T12:39:33.817 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:33.821 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:33.821 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:33.823 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:33.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:34.174 INFO:teuthology.orchestra.run.smithi116.stdout:generic/458 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T12:39:34.181 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/458 2024-02-18T12:39:34.181 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/458 2024-02-18T12:39:34.181 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:34.183 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:34.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:34.433 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-02-18T12:39:34.627 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:34.631 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:34.631 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:34.632 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:34.632 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:35.063 INFO:teuthology.orchestra.run.smithi116.stdout:generic/459 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:39:35.067 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/459 2024-02-18T12:39:35.067 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/459 2024-02-18T12:39:35.067 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:35.070 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:35.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:35.082 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-02-18T12:39:35.261 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:35.265 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:35.265 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:35.266 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:35.266 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:36.919 INFO:teuthology.orchestra.run.smithi116.stdout:generic/460 1s 2024-02-18T12:39:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/460 2024-02-18T12:39:36.924 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:36.926 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:36.935 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:36.936 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-02-18T12:39:37.110 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:37.114 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:37.114 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:37.115 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:37.115 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:37.407 INFO:teuthology.orchestra.run.smithi116.stdout:generic/461 [not run] ceph does not support shutdown 2024-02-18T12:39:37.413 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/461 2024-02-18T12:39:37.418 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/461 2024-02-18T12:39:37.418 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:37.418 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:37.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:37.450 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-02-18T12:39:37.712 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:37.718 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:37.718 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:37.724 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:37.724 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:37.995 INFO:teuthology.orchestra.run.smithi116.stdout:generic/462 [not run] mount 172.21.15.116:6789:/scratch with dax failed 2024-02-18T12:39:37.998 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/462 2024-02-18T12:39:37.999 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/462 2024-02-18T12:39:37.999 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:38.001 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:38.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:38.013 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-02-18T12:39:38.187 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:38.192 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:38.192 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:38.192 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:38.192 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:38.445 INFO:teuthology.orchestra.run.smithi116.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph 2024-02-18T12:39:38.449 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/463 2024-02-18T12:39:38.449 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/463 2024-02-18T12:39:38.449 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:39:38.451 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:39:38.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:39:38.515 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-02-18T12:39:38.884 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:39:38.888 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:39:38.889 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:39:38.890 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:39:38.890 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:34.317 INFO:teuthology.orchestra.run.smithi116.stdout:generic/464 56s 2024-02-18T12:40:34.321 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/464 2024-02-18T12:40:34.321 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:34.323 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:34.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:34.332 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-02-18T12:40:34.524 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:34.528 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:34.528 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:34.529 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:34.530 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:38.728 INFO:teuthology.orchestra.run.smithi116.stdout:generic/465 4s 2024-02-18T12:40:38.735 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/465 2024-02-18T12:40:38.735 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:38.738 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:38.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:38.750 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-02-18T12:40:39.067 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:39.071 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:39.072 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:39.073 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:39.073 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:39.284 INFO:teuthology.orchestra.run.smithi116.stdout:generic/466 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:40:39.288 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/466 2024-02-18T12:40:39.288 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/466 2024-02-18T12:40:39.288 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:39.291 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:39.303 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:39.304 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-02-18T12:40:39.483 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:39.487 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:39.487 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:39.488 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:39.488 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:47.304 INFO:teuthology.orchestra.run.smithi116.stdout:generic/467 8s 2024-02-18T12:40:47.345 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/467 2024-02-18T12:40:47.345 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:47.348 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:47.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:47.526 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-02-18T12:40:48.591 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:48.595 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:48.595 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:48.596 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:48.596 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:48.906 INFO:teuthology.orchestra.run.smithi116.stdout:generic/468 [not run] ceph does not support shutdown 2024-02-18T12:40:48.913 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/468 2024-02-18T12:40:48.913 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/468 2024-02-18T12:40:48.913 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:48.915 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:48.926 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:48.926 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-02-18T12:40:49.109 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:49.113 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:49.113 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:49.114 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:49.114 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:49.328 INFO:teuthology.orchestra.run.smithi116.stdout:generic/469 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T12:40:49.332 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/469 2024-02-18T12:40:49.332 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/469 2024-02-18T12:40:49.332 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:49.334 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:49.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:49.349 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-02-18T12:40:49.520 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:49.524 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:49.524 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:49.525 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:49.526 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:49.797 INFO:teuthology.orchestra.run.smithi116.stdout:generic/470 [not run] This test requires a valid $LOGWRITES_DEV 2024-02-18T12:40:49.801 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/470 2024-02-18T12:40:49.801 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/470 2024-02-18T12:40:49.801 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:40:49.803 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:40:49.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:40:49.815 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-02-18T12:40:50.000 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:40:50.004 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:40:50.004 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:40:50.005 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:40:50.005 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:13.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:41:13.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:41:28.761 INFO:teuthology.orchestra.run.smithi116.stdout:generic/471 38s 2024-02-18T12:41:28.774 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/471 2024-02-18T12:41:28.774 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:41:28.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:28.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:41:28.877 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-02-18T12:41:29.320 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:41:29.325 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:41:29.331 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:41:29.334 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:41:29.334 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:29.776 INFO:teuthology.orchestra.run.smithi116.stdout:generic/472 [not run] swapfiles are not supported 2024-02-18T12:41:29.779 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/472 2024-02-18T12:41:29.779 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/472 2024-02-18T12:41:29.779 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:41:29.782 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:29.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:41:29.794 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-02-18T12:41:29.971 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:41:29.975 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:41:29.975 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:41:29.976 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:41:29.976 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:30.243 INFO:teuthology.orchestra.run.smithi116.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-02-18T12:41:30.247 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/473 2024-02-18T12:41:30.247 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/473 2024-02-18T12:41:30.247 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:41:30.249 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:30.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:41:30.263 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-02-18T12:41:30.435 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:41:30.439 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:41:30.439 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:41:30.440 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:41:30.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:30.704 INFO:teuthology.orchestra.run.smithi116.stdout:generic/474 [not run] ceph does not support shutdown 2024-02-18T12:41:30.710 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/474 2024-02-18T12:41:30.710 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/474 2024-02-18T12:41:30.711 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:41:30.712 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:30.723 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:41:30.723 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-02-18T12:41:30.895 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:41:30.899 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:41:30.899 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:41:30.900 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:41:30.901 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:31.152 INFO:teuthology.orchestra.run.smithi116.stdout:generic/475 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T12:41:31.155 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/475 2024-02-18T12:41:31.156 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/475 2024-02-18T12:41:31.156 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T12:41:31.158 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:31.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T12:41:31.170 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-02-18T12:41:31.354 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T12:41:31.358 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T12:41:31.358 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T12:41:31.359 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T12:41:31.359 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T12:41:44.010 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:41:44.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:42:14.102 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:42:14.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:42:14.145 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T12:42:14.143+0000 7f627de19640 -1 Fail to open '/proc/581540/cmdline' error = (2) No such file or directory 2024-02-18T12:42:14.146 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T12:42:14.143+0000 7f348c2aa640 -1 Fail to open '/proc/581540/cmdline' error = (2) No such file or directory 2024-02-18T12:42:14.146 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T12:42:14.143+0000 7f348c2aa640 -1 received signal: Hangup from (PID: 581540) UID: 0 2024-02-18T12:42:14.146 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T12:42:14.143+0000 7f627de19640 -1 received signal: Hangup from (PID: 581540) UID: 0 2024-02-18T12:42:14.146 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T12:42:14.144+0000 7f9b36a5c640 -1 Fail to open '/proc/581540/cmdline' error = (2) No such file or directory 2024-02-18T12:42:14.146 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T12:42:14.144+0000 7f9b36a5c640 -1 received signal: Hangup from (PID: 581540) UID: 0 2024-02-18T12:42:14.147 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T12:42:14.144+0000 7f5c53248640 -1 Fail to open '/proc/581540/cmdline' error = (2) No such file or directory 2024-02-18T12:42:14.147 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T12:42:14.144+0000 7f5c53248640 -1 received signal: Hangup from (PID: 581540) UID: 0 2024-02-18T12:48:14.386 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:48:14.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T12:48:14.638 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T12:48:14.637+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50785) UID: 0 2024-02-18T12:48:14.639 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T12:48:14.636+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50785) UID: 0 2024-02-18T12:48:14.649 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T12:48:14.648+0000 7fed528fa640 -1 Fail to open '/proc/50785/cmdline' error = (2) No such file or directory 2024-02-18T12:48:14.649 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T12:48:14.648+0000 7fed528fa640 -1 received signal: Hangup from (PID: 50785) UID: 0 2024-02-18T12:48:14.653 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T12:48:14.639+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 50785) UID: 0 2024-02-18T12:48:15.472 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T12:48:15.471+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 581952) UID: 0 2024-02-18T12:48:15.472 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T12:48:15.471+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 581952) UID: 0 2024-02-18T12:53:56.464 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T12:53:56.461+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582048) UID: 0 2024-02-18T12:53:56.465 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T12:53:56.464+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582048) UID: 0 2024-02-18T12:53:56.466 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T12:53:56.463+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582048) UID: 0 2024-02-18T12:53:56.484 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T12:53:56.468+0000 7f9b36a5c640 -1 Fail to open '/proc/582048/cmdline' error = (2) No such file or directory 2024-02-18T12:53:56.484 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T12:53:56.468+0000 7f9b36a5c640 -1 received signal: Hangup from (PID: 582048) UID: 0 2024-02-18T13:10:50.019 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T13:10:50.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T13:10:51.669 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T13:10:51.670+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 582247) UID: 0 2024-02-18T13:10:51.670 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T13:10:51.670+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 582247) UID: 0 2024-02-18T13:10:51.801 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T13:10:51.801+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51065) UID: 0 2024-02-18T13:10:51.802 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T13:10:51.801+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51065) UID: 0 2024-02-18T13:10:51.802 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T13:10:51.801+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51065) UID: 0 2024-02-18T13:10:51.807 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T13:10:51.801+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51065) UID: 0 2024-02-18T13:19:59.918 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T13:19:59.917+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582346) UID: 0 2024-02-18T13:19:59.919 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T13:19:59.919+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582346) UID: 0 2024-02-18T13:19:59.919 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T13:19:59.915+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582346) UID: 0 2024-02-18T13:19:59.923 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T13:19:59.920+0000 7f627de19640 -1 Fail to open '/proc/582346/cmdline' error = (2) No such file or directory 2024-02-18T13:19:59.923 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T13:19:59.920+0000 7f627de19640 -1 received signal: Hangup from (PID: 582346) UID: 0 2024-02-18T13:31:37.688 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T13:31:37.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T13:31:37.934 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T13:31:37.933+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51260) UID: 0 2024-02-18T13:31:37.934 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T13:31:37.933+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51260) UID: 0 2024-02-18T13:31:37.935 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T13:31:37.933+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51260) UID: 0 2024-02-18T13:31:37.935 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T13:31:37.933+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51260) UID: 0 2024-02-18T13:31:38.264 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T13:31:38.265+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 582553) UID: 0 2024-02-18T13:31:38.264 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T13:31:38.265+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 582553) UID: 0 2024-02-18T13:40:41.252 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T13:40:41.251+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582643) UID: 0 2024-02-18T13:40:41.252 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T13:40:41.251+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582643) UID: 0 2024-02-18T13:40:41.253 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T13:40:41.251+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582643) UID: 0 2024-02-18T13:40:41.253 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T13:40:41.251+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582643) UID: 0 2024-02-18T13:52:29.864 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T13:52:29.929 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T13:52:30.204 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T13:52:30.202+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51585) UID: 0 2024-02-18T13:52:30.204 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T13:52:30.202+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51585) UID: 0 2024-02-18T13:52:30.205 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T13:52:30.202+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51585) UID: 0 2024-02-18T13:52:30.205 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T13:52:30.202+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51585) UID: 0 2024-02-18T13:52:30.208 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T13:52:30.207+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 582814) UID: 0 2024-02-18T13:52:30.208 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T13:52:30.207+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 582814) UID: 0 2024-02-18T14:05:23.155 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T14:05:23.151+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 583056) UID: 0 2024-02-18T14:05:23.180 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T14:05:23.153+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 583056) UID: 0 2024-02-18T14:05:23.180 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T14:05:23.153+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 583056) UID: 0 2024-02-18T14:05:23.181 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T14:05:23.154+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 583056) UID: 0 2024-02-18T14:16:05.992 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:16:05.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:16:06.255 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T14:16:06.253+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51975) UID: 0 2024-02-18T14:16:06.263 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T14:16:06.256+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51975) UID: 0 2024-02-18T14:16:06.263 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T14:16:06.252+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51975) UID: 0 2024-02-18T14:16:06.265 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T14:16:06.252+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 51975) UID: 0 2024-02-18T14:16:06.332 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T14:16:06.330+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 583189) UID: 0 2024-02-18T14:16:06.336 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T14:16:06.334+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 583189) UID: 0 2024-02-18T14:16:43.335 INFO:teuthology.orchestra.run.smithi116.stdout:generic/476 5712s 2024-02-18T14:16:43.579 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/476 2024-02-18T14:16:43.579 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:16:43.583 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:16:43.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:16:43.615 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-02-18T14:16:46.280 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:16:46.284 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:16:46.284 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:16:46.285 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:16:46.286 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:22.821 INFO:teuthology.orchestra.run.smithi116.stdout:generic/477 32s 2024-02-18T14:23:22.967 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/477 2024-02-18T14:23:22.967 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:22.969 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:23.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:23.181 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-02-18T14:23:26.618 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:26.622 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:26.622 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:26.623 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:26.624 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:37.710 INFO:teuthology.orchestra.run.smithi116.stdout:generic/478 11s 2024-02-18T14:23:37.719 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/478 2024-02-18T14:23:37.719 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:37.722 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:37.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:37.751 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-02-18T14:23:38.002 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:38.007 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:38.007 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:38.009 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:38.009 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:38.622 INFO:teuthology.orchestra.run.smithi116.stdout:generic/479 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:38.627 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/479 2024-02-18T14:23:38.627 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/479 2024-02-18T14:23:38.628 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:38.630 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:38.650 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:38.650 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-02-18T14:23:38.943 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:38.948 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:38.948 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:38.953 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:38.953 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:39.302 INFO:teuthology.orchestra.run.smithi116.stdout:generic/480 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:39.308 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/480 2024-02-18T14:23:39.308 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/480 2024-02-18T14:23:39.308 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:39.312 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:39.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:39.349 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-02-18T14:23:39.618 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:39.623 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:39.624 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:39.626 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:39.626 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:39.952 INFO:teuthology.orchestra.run.smithi116.stdout:generic/481 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:39.956 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/481 2024-02-18T14:23:39.956 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/481 2024-02-18T14:23:39.956 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:39.963 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:39.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:39.993 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-02-18T14:23:40.252 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:40.257 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:40.257 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:40.258 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:40.258 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:40.637 INFO:teuthology.orchestra.run.smithi116.stdout:generic/482 [not run] This test requires a valid $LOGWRITES_DEV 2024-02-18T14:23:40.643 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/482 2024-02-18T14:23:40.643 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/482 2024-02-18T14:23:40.643 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:40.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:40.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:40.665 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-02-18T14:23:40.937 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:40.943 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:40.943 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:40.944 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:41.258 INFO:teuthology.orchestra.run.smithi116.stdout:generic/483 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:41.262 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/483 2024-02-18T14:23:41.263 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/483 2024-02-18T14:23:41.263 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:41.266 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:41.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:41.284 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-02-18T14:23:41.551 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:41.555 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:41.557 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:41.557 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:41.930 INFO:teuthology.orchestra.run.smithi116.stdout:generic/484 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:41.937 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/484 2024-02-18T14:23:41.937 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/484 2024-02-18T14:23:41.938 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:41.940 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:41.962 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:41.962 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-02-18T14:23:42.212 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:42.220 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:42.220 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:42.220 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:42.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:42.543 INFO:teuthology.orchestra.run.smithi116.stdout:generic/485 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T14:23:42.549 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/485 2024-02-18T14:23:42.549 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/485 2024-02-18T14:23:42.549 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:42.552 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:42.590 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:42.590 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-02-18T14:23:42.869 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:42.873 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:42.874 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:42.875 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:42.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:43.498 INFO:teuthology.orchestra.run.smithi116.stdout:generic/486 0s 2024-02-18T14:23:43.505 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/486 2024-02-18T14:23:43.505 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:43.508 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:43.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:43.518 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-02-18T14:23:43.771 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:43.777 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:43.778 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:43.780 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:43.780 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:44.137 INFO:teuthology.orchestra.run.smithi116.stdout:generic/487 [not run] This test requires a valid $SCRATCH_LOGDEV 2024-02-18T14:23:44.142 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/487 2024-02-18T14:23:44.142 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/487 2024-02-18T14:23:44.142 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:44.146 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:44.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:44.167 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-02-18T14:23:44.441 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:44.448 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:44.448 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:44.449 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:44.449 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:44.772 INFO:teuthology.orchestra.run.smithi116.stdout:generic/488 [not run] Filesystem ceph not supported in _scratch_mkfs_sized 2024-02-18T14:23:44.777 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/488 2024-02-18T14:23:44.777 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/488 2024-02-18T14:23:44.777 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:44.779 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:44.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:44.811 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-02-18T14:23:45.076 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:45.084 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:45.087 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:45.088 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:45.088 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:45.404 INFO:teuthology.orchestra.run.smithi116.stdout:generic/489 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:45.409 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/489 2024-02-18T14:23:45.410 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/489 2024-02-18T14:23:45.410 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:45.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:45.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:45.439 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-02-18T14:23:45.704 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:45.710 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:45.711 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:45.713 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:45.713 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:48.075 INFO:teuthology.orchestra.run.smithi116.stdout:generic/490 2s 2024-02-18T14:23:48.092 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/490 2024-02-18T14:23:48.092 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:48.095 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:48.181 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:48.182 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-02-18T14:23:48.569 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:48.575 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:48.575 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:48.577 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:48.577 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:49.331 INFO:teuthology.orchestra.run.smithi116.stdout:generic/491 [not run] ceph does not support freezing 2024-02-18T14:23:49.336 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/491 2024-02-18T14:23:49.336 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/491 2024-02-18T14:23:49.336 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:49.340 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:49.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:49.363 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-02-18T14:23:49.644 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:49.648 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:49.648 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:49.649 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:49.649 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:50.050 INFO:teuthology.orchestra.run.smithi116.stdout:generic/492 [not run] xfs_io label failed (old kernel/wrong fs?) 2024-02-18T14:23:50.057 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/492 2024-02-18T14:23:50.057 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/492 2024-02-18T14:23:50.057 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:50.061 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:50.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:50.079 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-02-18T14:23:50.338 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:50.344 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:50.344 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:50.346 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:50.346 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:51.198 INFO:teuthology.orchestra.run.smithi116.stdout:generic/493 [not run] swapfiles are not supported 2024-02-18T14:23:51.203 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/493 2024-02-18T14:23:51.204 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/493 2024-02-18T14:23:51.204 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:51.207 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:51.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:51.235 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-02-18T14:23:51.499 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:51.503 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:51.504 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:51.505 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:51.505 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:51.971 INFO:teuthology.orchestra.run.smithi116.stdout:generic/494 [not run] swapfiles are not supported 2024-02-18T14:23:51.977 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/494 2024-02-18T14:23:51.977 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/494 2024-02-18T14:23:51.977 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:51.980 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:52.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:52.017 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-02-18T14:23:52.276 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:52.282 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:52.283 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:52.284 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:52.284 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:52.764 INFO:teuthology.orchestra.run.smithi116.stdout:generic/495 [not run] swapfiles are not supported 2024-02-18T14:23:52.769 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/495 2024-02-18T14:23:52.769 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/495 2024-02-18T14:23:52.769 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:52.772 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:52.797 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:52.797 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-02-18T14:23:53.033 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:53.039 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:53.040 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:53.041 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:53.041 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:53.527 INFO:teuthology.orchestra.run.smithi116.stdout:generic/496 [not run] swapfiles are not supported 2024-02-18T14:23:53.532 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/496 2024-02-18T14:23:53.532 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/496 2024-02-18T14:23:53.532 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:53.535 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:53.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:53.561 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-02-18T14:23:53.837 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:53.842 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:53.843 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:53.844 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:53.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:54.328 INFO:teuthology.orchestra.run.smithi116.stdout:generic/497 [not run] swapfiles are not supported 2024-02-18T14:23:54.334 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/497 2024-02-18T14:23:54.334 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/497 2024-02-18T14:23:54.334 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:54.337 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:54.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:54.362 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-02-18T14:23:54.650 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:54.654 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:54.654 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:54.655 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:54.655 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:54.991 INFO:teuthology.orchestra.run.smithi116.stdout:generic/498 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:54.997 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/498 2024-02-18T14:23:54.997 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/498 2024-02-18T14:23:54.997 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:55.000 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:55.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:55.034 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-02-18T14:23:55.300 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:55.304 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:55.305 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:55.307 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:55.307 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:55.636 INFO:teuthology.orchestra.run.smithi116.stdout:generic/499 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T14:23:55.641 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/499 2024-02-18T14:23:55.641 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/499 2024-02-18T14:23:55.641 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:55.643 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:55.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:55.670 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-02-18T14:23:55.918 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:55.923 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:55.924 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:55.925 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:55.925 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:56.264 INFO:teuthology.orchestra.run.smithi116.stdout:generic/500 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:56.269 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/500 2024-02-18T14:23:56.269 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/500 2024-02-18T14:23:56.269 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:56.271 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:56.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:56.296 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-02-18T14:23:56.563 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:56.567 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:56.567 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:56.568 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:56.568 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:57.096 INFO:teuthology.orchestra.run.smithi116.stdout:generic/501 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T14:23:57.109 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/501 2024-02-18T14:23:57.109 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/501 2024-02-18T14:23:57.109 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:57.111 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:57.347 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:57.348 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-02-18T14:23:57.672 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:57.677 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:57.678 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:57.679 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:57.679 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:58.025 INFO:teuthology.orchestra.run.smithi116.stdout:generic/502 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:23:58.030 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/502 2024-02-18T14:23:58.030 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/502 2024-02-18T14:23:58.030 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:58.033 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:58.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:58.059 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-02-18T14:23:58.339 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:58.348 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:58.349 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:58.350 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:58.350 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:58.665 INFO:teuthology.orchestra.run.smithi116.stdout:generic/503 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T14:23:58.669 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/503 2024-02-18T14:23:58.669 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/503 2024-02-18T14:23:58.670 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:58.672 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:58.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:58.709 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-02-18T14:23:58.966 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:58.971 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:58.971 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:58.973 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:58.973 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:59.352 INFO:teuthology.orchestra.run.smithi116.stdout:generic/504 0s 2024-02-18T14:23:59.360 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/504 2024-02-18T14:23:59.360 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:23:59.363 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:23:59.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:23:59.406 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-02-18T14:23:59.652 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:23:59.657 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:23:59.657 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:23:59.659 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:23:59.659 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:00.082 INFO:teuthology.orchestra.run.smithi116.stdout:generic/505 [not run] ceph does not support shutdown 2024-02-18T14:24:00.109 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/505 2024-02-18T14:24:00.109 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/505 2024-02-18T14:24:00.109 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:00.113 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:00.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:00.137 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-02-18T14:24:00.419 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:00.423 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:00.423 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:00.425 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:00.425 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:00.738 INFO:teuthology.orchestra.run.smithi116.stdout:generic/506 [not run] disk quotas not supported by this filesystem type: ceph 2024-02-18T14:24:00.743 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/506 2024-02-18T14:24:00.743 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/506 2024-02-18T14:24:00.743 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:00.746 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:00.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:00.768 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-02-18T14:24:01.046 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:01.051 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:01.052 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:01.053 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:01.054 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:01.425 INFO:teuthology.orchestra.run.smithi116.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-02-18T14:24:01.429 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/507 2024-02-18T14:24:01.429 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/507 2024-02-18T14:24:01.429 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:01.435 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:01.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:01.468 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-02-18T14:24:01.739 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:01.745 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:01.745 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:01.746 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:01.747 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:02.053 INFO:teuthology.orchestra.run.smithi116.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph 2024-02-18T14:24:02.058 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/508 2024-02-18T14:24:02.058 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/508 2024-02-18T14:24:02.058 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:02.061 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:02.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:02.085 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-02-18T14:24:02.358 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:02.364 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:02.364 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:02.365 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:02.365 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:02.688 INFO:teuthology.orchestra.run.smithi116.stdout:generic/509 [not run] O_TMPFILE is not supported 2024-02-18T14:24:02.692 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/509 2024-02-18T14:24:02.693 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/509 2024-02-18T14:24:02.693 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:02.696 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:02.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:02.716 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-02-18T14:24:02.986 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:02.991 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:02.991 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:02.993 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:02.993 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:03.302 INFO:teuthology.orchestra.run.smithi116.stdout:generic/510 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:24:03.307 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/510 2024-02-18T14:24:03.307 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/510 2024-02-18T14:24:03.307 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:03.310 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:03.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:03.333 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-02-18T14:24:03.627 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:03.632 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:03.633 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:03.638 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:03.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:03.988 INFO:teuthology.orchestra.run.smithi116.stdout:generic/511 [not run] xfs_io falloc -k failed (old kernel/wrong fs?) 2024-02-18T14:24:03.995 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/511 2024-02-18T14:24:03.996 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/511 2024-02-18T14:24:03.996 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:03.997 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:04.036 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:04.036 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-02-18T14:24:04.316 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:04.321 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:04.321 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:04.322 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:04.323 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:04.644 INFO:teuthology.orchestra.run.smithi116.stdout:generic/512 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T14:24:04.649 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/512 2024-02-18T14:24:04.649 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/512 2024-02-18T14:24:04.649 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:04.651 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:04.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:04.681 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-02-18T14:24:04.963 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:04.969 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:04.969 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:04.970 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:04.970 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:05.479 INFO:teuthology.orchestra.run.smithi116.stdout:generic/513 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T14:24:05.490 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/513 2024-02-18T14:24:05.491 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/513 2024-02-18T14:24:05.491 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:05.494 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:05.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:05.753 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-02-18T14:24:06.100 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:06.104 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:06.105 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:06.106 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:06.106 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:06.661 INFO:teuthology.orchestra.run.smithi116.stdout:generic/514 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T14:24:06.667 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/514 2024-02-18T14:24:06.667 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/514 2024-02-18T14:24:06.667 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:06.670 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:06.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:06.966 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-02-18T14:24:07.291 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:07.296 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:07.296 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:07.298 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:07.298 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:07.801 INFO:teuthology.orchestra.run.smithi116.stdout:generic/515 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T14:24:07.808 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/515 2024-02-18T14:24:07.808 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/515 2024-02-18T14:24:07.808 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:07.811 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:08.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:08.085 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-02-18T14:24:08.402 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:08.408 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:08.409 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:08.410 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:08.410 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:08.780 INFO:teuthology.orchestra.run.smithi116.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph 2024-02-18T14:24:08.785 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/516 2024-02-18T14:24:08.785 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/516 2024-02-18T14:24:08.785 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:08.789 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:08.862 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:08.862 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-02-18T14:24:09.290 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:09.296 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:09.297 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:09.298 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:09.298 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:09.781 INFO:teuthology.orchestra.run.smithi116.stdout:generic/517 [not run] Dedupe not supported by scratch filesystem type: ceph 2024-02-18T14:24:09.827 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/517 2024-02-18T14:24:09.827 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/517 2024-02-18T14:24:09.828 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:09.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:10.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:10.052 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-02-18T14:24:10.389 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:10.395 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:10.395 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:10.397 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:10.397 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:10.920 INFO:teuthology.orchestra.run.smithi116.stdout:generic/518 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T14:24:10.926 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/518 2024-02-18T14:24:10.927 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/518 2024-02-18T14:24:10.927 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:10.930 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:11.190 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:11.190 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-02-18T14:24:11.515 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:11.523 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:11.523 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:11.525 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:11.525 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:11.916 INFO:teuthology.orchestra.run.smithi116.stdout:generic/519 [not run] FIBMAP not supported by this filesystem 2024-02-18T14:24:11.921 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/519 2024-02-18T14:24:11.921 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/519 2024-02-18T14:24:11.921 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:11.926 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:11.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:11.991 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-02-18T14:24:12.476 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:12.484 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:12.484 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:12.485 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:12.485 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:12.796 INFO:teuthology.orchestra.run.smithi116.stdout:generic/520 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T14:24:12.801 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/520 2024-02-18T14:24:12.801 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/520 2024-02-18T14:24:12.801 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T14:24:12.807 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:24:12.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T14:24:12.827 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-02-18T14:24:13.089 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T14:24:13.094 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T14:24:13.095 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T14:24:13.096 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T14:24:13.096 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T14:30:49.848 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T14:30:49.845+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604768) UID: 0 2024-02-18T14:30:49.849 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T14:30:49.845+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604768) UID: 0 2024-02-18T14:30:49.850 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T14:30:49.845+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604768) UID: 0 2024-02-18T14:30:49.850 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T14:30:49.845+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604768) UID: 0 2024-02-18T14:38:41.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:38:41.031 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:38:41.311 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T14:38:41.308+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 604793) UID: 0 2024-02-18T14:38:41.311 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T14:38:41.308+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 604793) UID: 0 2024-02-18T14:38:41.355 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T14:38:41.351+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52015) UID: 0 2024-02-18T14:38:41.356 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T14:38:41.351+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52015) UID: 0 2024-02-18T14:38:41.356 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T14:38:41.351+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52015) UID: 0 2024-02-18T14:38:41.356 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T14:38:41.351+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52015) UID: 0 2024-02-18T14:43:38.754 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T14:43:38.751+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604798) UID: 0 2024-02-18T14:43:38.755 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T14:43:38.751+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604798) UID: 0 2024-02-18T14:43:38.756 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T14:43:38.751+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604798) UID: 0 2024-02-18T14:43:38.756 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T14:43:38.751+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604798) UID: 0 2024-02-18T14:55:30.909 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:55:30.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:55:31.157 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T14:55:31.155+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604826) UID: 0 2024-02-18T14:55:31.158 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T14:55:31.155+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604826) UID: 0 2024-02-18T14:55:31.158 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T14:55:31.155+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604826) UID: 0 2024-02-18T14:55:31.158 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T14:55:31.155+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604826) UID: 0 2024-02-18T14:59:59.648 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:59:59.652 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T14:59:59.727 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T14:59:59.724+0000 7f2d640d1640 -1 Fail to open '/proc/52074/cmdline' error = (2) No such file or directory 2024-02-18T14:59:59.727 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T14:59:59.724+0000 7f2d640d1640 -1 received signal: Hangup from (PID: 52074) UID: 0 2024-02-18T14:59:59.728 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T14:59:59.724+0000 7ff399284640 -1 Fail to open '/proc/52074/cmdline' error = (2) No such file or directory 2024-02-18T14:59:59.728 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T14:59:59.724+0000 7ff399284640 -1 received signal: Hangup from (PID: 52074) UID: 0 2024-02-18T14:59:59.728 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T14:59:59.724+0000 7fed528fa640 -1 Fail to open '/proc/52074/cmdline' error = (2) No such file or directory 2024-02-18T14:59:59.728 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T14:59:59.724+0000 7fed528fa640 -1 received signal: Hangup from (PID: 52074) UID: 0 2024-02-18T14:59:59.728 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T14:59:59.724+0000 7fe4fc5cc640 -1 Fail to open '/proc/52074/cmdline' error = (2) No such file or directory 2024-02-18T14:59:59.729 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T14:59:59.724+0000 7fe4fc5cc640 -1 received signal: Hangup from (PID: 52074) UID: 0 2024-02-18T15:08:16.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:08:16.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:08:16.734 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T15:08:16.732+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52112) UID: 0 2024-02-18T15:08:16.734 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T15:08:16.732+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52112) UID: 0 2024-02-18T15:08:16.734 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:08:16.732+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52112) UID: 0 2024-02-18T15:08:16.735 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T15:08:16.732+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52112) UID: 0 2024-02-18T15:13:38.672 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:13:38.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:13:38.909 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T15:13:38.907+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604903) UID: 0 2024-02-18T15:13:38.910 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T15:13:38.907+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604903) UID: 0 2024-02-18T15:13:38.910 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T15:13:38.907+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604903) UID: 0 2024-02-18T15:13:38.910 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T15:13:38.907+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604903) UID: 0 2024-02-18T15:17:21.931 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:17:21.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:17:22.068 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T15:17:22.065+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604925) UID: 0 2024-02-18T15:17:22.068 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T15:17:22.065+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604925) UID: 0 2024-02-18T15:17:22.068 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T15:17:22.065+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604925) UID: 0 2024-02-18T15:17:22.069 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T15:17:22.065+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604925) UID: 0 2024-02-18T15:24:34.003 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:24:34.031 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:24:34.332 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T15:24:34.337+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52188) UID: 0 2024-02-18T15:24:34.333 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T15:24:34.337+0000 7fe4fc5cc640 -1 received signal: Hangup from (PID: 52188) UID: 0 2024-02-18T15:24:34.335 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T15:24:34.340+0000 7f2d640d1640 -1 Fail to open '/proc/52188/cmdline' error = (2) No such file or directory 2024-02-18T15:24:34.335 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T15:24:34.340+0000 7f2d640d1640 -1 received signal: Hangup from (PID: 52188) UID: 0 2024-02-18T15:24:34.336 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:24:34.341+0000 7fed528fa640 -1 Fail to open '/proc/52188/cmdline' error = (2) No such file or directory 2024-02-18T15:24:34.336 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:24:34.341+0000 7fed528fa640 -1 received signal: Hangup from (PID: 52188) UID: 0 2024-02-18T15:28:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.6.log.1: 2024-02-18T15:28:23.916 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: stdin: file size changed while zipping 2024-02-18T15:33:34.643 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:33:34.666 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:33:34.920 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T15:33:34.930+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52216) UID: 0 2024-02-18T15:33:34.920 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:33:34.930+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52216) UID: 0 2024-02-18T15:33:34.921 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T15:33:34.930+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52216) UID: 0 2024-02-18T15:33:34.921 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T15:33:34.931+0000 7ff399284640 -1 Fail to open '/proc/52216/cmdline' error = (2) No such file or directory 2024-02-18T15:33:34.921 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T15:33:34.931+0000 7ff399284640 -1 received signal: Hangup from (PID: 52216) UID: 0 2024-02-18T15:38:17.499 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:38:17.503 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:38:17.960 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T15:38:17.973+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604995) UID: 0 2024-02-18T15:38:17.961 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T15:38:17.973+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604995) UID: 0 2024-02-18T15:38:17.961 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T15:38:17.973+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604995) UID: 0 2024-02-18T15:38:17.962 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T15:38:17.973+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 604995) UID: 0 2024-02-18T15:42:03.219 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:42:03.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:42:03.263 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T15:42:03.276+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52265) UID: 0 2024-02-18T15:42:03.263 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T15:42:03.276+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52265) UID: 0 2024-02-18T15:42:03.264 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:42:03.276+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52265) UID: 0 2024-02-18T15:42:03.264 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T15:42:03.276+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52265) UID: 0 2024-02-18T15:46:24.852 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:46:24.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:46:25.907 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:46:25.922+0000 7fed528fa640 -1 Fail to open '/proc/52288/cmdline' error = (2) No such file or directory 2024-02-18T15:46:25.907 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T15:46:25.922+0000 7fed528fa640 -1 received signal: Hangup from (PID: 52288) UID: 0 2024-02-18T15:46:25.907 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T15:46:25.921+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52288) UID: 0 2024-02-18T15:46:25.907 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T15:46:25.921+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52288) UID: 0 2024-02-18T15:46:25.908 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T15:46:25.923+0000 7fe4fc5cc640 -1 Fail to open '/proc/52288/cmdline' error = (2) No such file or directory 2024-02-18T15:46:25.908 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T15:46:25.923+0000 7fe4fc5cc640 -1 received signal: Hangup from (PID: 52288) UID: 0 2024-02-18T15:49:18.670 INFO:teuthology.orchestra.run.smithi116.stdout:generic/521 5105s 2024-02-18T15:49:18.711 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/521 2024-02-18T15:49:18.711 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T15:49:18.714 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T15:49:18.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T15:49:18.768 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-02-18T15:49:19.357 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T15:49:19.361 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T15:49:19.362 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T15:49:19.363 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T15:49:19.363 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T15:50:36.398 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:50:36.401 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:50:36.443 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T15:50:36.460+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605538) UID: 0 2024-02-18T15:50:36.444 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T15:50:36.460+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605538) UID: 0 2024-02-18T15:50:36.444 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T15:50:36.460+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605538) UID: 0 2024-02-18T15:50:36.444 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T15:50:36.460+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605538) UID: 0 2024-02-18T15:54:30.034 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:54:30.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T15:54:30.288 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T15:54:30.302+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 605560) UID: 0 2024-02-18T15:54:30.289 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T15:54:30.302+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 605560) UID: 0 2024-02-18T15:57:45.699 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T15:57:45.705+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605565) UID: 0 2024-02-18T15:57:45.699 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T15:57:45.706+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605565) UID: 0 2024-02-18T15:57:45.700 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T15:57:45.707+0000 7f627de19640 -1 Fail to open '/proc/605565/cmdline' error = (2) No such file or directory 2024-02-18T15:57:45.700 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T15:57:45.707+0000 7f627de19640 -1 received signal: Hangup from (PID: 605565) UID: 0 2024-02-18T15:57:45.700 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T15:57:45.705+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605565) UID: 0 2024-02-18T16:02:04.435 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:02:04.441 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:02:04.567 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:02:04.570+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52375) UID: 0 2024-02-18T16:02:04.568 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:02:04.570+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52375) UID: 0 2024-02-18T16:02:04.568 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:02:04.570+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52375) UID: 0 2024-02-18T16:02:04.568 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:02:04.570+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52375) UID: 0 2024-02-18T16:06:44.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:06:44.890 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:06:44.931 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:06:44.930+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605621) UID: 0 2024-02-18T16:06:44.932 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T16:06:44.930+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605621) UID: 0 2024-02-18T16:06:44.933 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T16:06:44.930+0000 7f627de19640 -1 received signal: Hangup from (PID: 605621) UID: 0 2024-02-18T16:06:44.933 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:06:44.931+0000 7f348c2aa640 -1 Fail to open '/proc/605621/cmdline' error = (2) No such file or directory 2024-02-18T16:06:44.933 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:06:44.931+0000 7f348c2aa640 -1 received signal: Hangup from (PID: 605621) UID: 0 2024-02-18T16:06:45.996 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:06:45.995+0000 7fe4fc5cc640 -1 Fail to open '/proc/52400/cmdline' error = (2) No such file or directory 2024-02-18T16:06:45.997 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:06:45.996+0000 7fe4fc5cc640 -1 received signal: Hangup from (PID: 52400) UID: 0 2024-02-18T16:06:45.997 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:06:45.995+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52400) UID: 0 2024-02-18T16:06:45.997 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:06:45.995+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52400) UID: 0 2024-02-18T16:06:45.998 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:06:45.996+0000 7ff399284640 -1 Fail to open '/proc/52400/cmdline' error = (2) No such file or directory 2024-02-18T16:06:45.998 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:06:45.996+0000 7ff399284640 -1 received signal: Hangup from (PID: 52400) UID: 0 2024-02-18T16:10:06.928 INFO:teuthology.orchestra.run.smithi143.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.6.log.1: 2024-02-18T16:10:06.928 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: stdin: file size changed while zipping 2024-02-18T16:14:43.270 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:14:43.273 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:14:43.313 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:14:43.308+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52427) UID: 0 2024-02-18T16:14:43.313 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:14:43.309+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52427) UID: 0 2024-02-18T16:14:43.314 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:14:43.309+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52427) UID: 0 2024-02-18T16:14:43.314 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:14:43.309+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52427) UID: 0 2024-02-18T16:18:47.751 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:18:47.755 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:18:47.979 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:18:47.973+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52452) UID: 0 2024-02-18T16:18:47.979 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:18:47.973+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52452) UID: 0 2024-02-18T16:18:47.980 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:18:47.973+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52452) UID: 0 2024-02-18T16:18:47.980 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:18:47.973+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52452) UID: 0 2024-02-18T16:22:50.601 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:22:50.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:22:50.794 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T16:22:50.788+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605687) UID: 0 2024-02-18T16:22:50.794 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:22:50.788+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605687) UID: 0 2024-02-18T16:22:50.796 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T16:22:50.788+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605687) UID: 0 2024-02-18T16:22:50.798 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:22:50.790+0000 7f5c53248640 -1 Fail to open '/proc/605687/cmdline' error = (2) No such file or directory 2024-02-18T16:22:50.798 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:22:50.790+0000 7f5c53248640 -1 received signal: Hangup from (PID: 605687) UID: 0 2024-02-18T16:26:44.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:26:44.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:27:14.986 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:27:14.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:27:45.074 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:27:45.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:28:15.357 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:28:15.360 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:28:45.431 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:28:45.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:28:45.474 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T16:28:45.469+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605782) UID: 0 2024-02-18T16:28:45.475 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:28:45.469+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605782) UID: 0 2024-02-18T16:28:45.475 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T16:28:45.469+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605782) UID: 0 2024-02-18T16:28:45.479 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:28:45.470+0000 7f348c2aa640 -1 Fail to open '/proc/605782/cmdline' error = (2) No such file or directory 2024-02-18T16:28:45.479 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:28:45.470+0000 7f348c2aa640 -1 received signal: Hangup from (PID: 605782) UID: 0 2024-02-18T16:32:20.558 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:32:20.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:32:20.608 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:32:20.601+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52591) UID: 0 2024-02-18T16:32:20.609 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:32:20.601+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52591) UID: 0 2024-02-18T16:32:20.609 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:32:20.601+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52591) UID: 0 2024-02-18T16:32:20.610 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:32:20.601+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52591) UID: 0 2024-02-18T16:36:25.669 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:36:25.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:36:55.904 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:36:55.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:37:25.970 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:37:25.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:37:56.059 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:37:56.062 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:38:26.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:38:26.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:38:56.279 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:38:56.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:39:26.343 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:39:26.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:39:26.444 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:39:26.436+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605935) UID: 0 2024-02-18T16:39:26.444 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T16:39:26.436+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605935) UID: 0 2024-02-18T16:39:26.444 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:39:26.437+0000 7f348c2aa640 -1 received signal: Hangup from (PID: 605935) UID: 0 2024-02-18T16:39:26.445 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T16:39:26.436+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 605935) UID: 0 2024-02-18T16:43:17.734 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:43:17.737 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:43:17.804 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:43:17.800+0000 7ff399284640 -1 received signal: Hangup from (PID: 52747) UID: 0 2024-02-18T16:43:17.804 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:43:17.800+0000 7fed528fa640 -1 received signal: Hangup from (PID: 52747) UID: 0 2024-02-18T16:43:17.805 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:43:17.800+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52747) UID: 0 2024-02-18T16:43:17.805 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:43:17.800+0000 7fe4fc5cc640 -1 received signal: Hangup from (PID: 52747) UID: 0 2024-02-18T16:47:09.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:47:09.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:47:09.764 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:47:09.761+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52771) UID: 0 2024-02-18T16:47:09.764 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:47:09.761+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52771) UID: 0 2024-02-18T16:47:09.765 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:47:09.761+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52771) UID: 0 2024-02-18T16:47:09.765 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:47:09.761+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 52771) UID: 0 2024-02-18T16:50:08.245 INFO:teuthology.orchestra.run.smithi116.stdout:generic/522 3649s 2024-02-18T16:50:08.302 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/522 2024-02-18T16:50:08.302 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:08.305 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:08.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:08.358 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-02-18T16:50:08.871 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:08.876 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:08.876 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:08.878 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:08.878 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:09.411 INFO:teuthology.orchestra.run.smithi116.stdout:generic/523 0s 2024-02-18T16:50:09.420 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/523 2024-02-18T16:50:09.420 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:09.420 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:09.430 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:09.430 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-02-18T16:50:09.662 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:09.667 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:09.668 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:09.669 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:09.669 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:14.672 INFO:teuthology.orchestra.run.smithi116.stdout:generic/524 5s 2024-02-18T16:50:14.680 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/524 2024-02-18T16:50:14.680 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:14.683 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:14.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:14.694 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-02-18T16:50:14.931 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:14.936 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:14.937 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:14.938 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:14.938 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:15.368 INFO:teuthology.orchestra.run.smithi116.stdout:generic/525 [not run] filesystem does not support huge file size 2024-02-18T16:50:15.395 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/525 2024-02-18T16:50:15.396 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/525 2024-02-18T16:50:15.396 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:15.398 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:15.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:15.427 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-02-18T16:50:15.668 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:15.673 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:15.674 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:15.676 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:15.676 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:16.076 INFO:teuthology.orchestra.run.smithi116.stdout:generic/526 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T16:50:16.081 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/526 2024-02-18T16:50:16.081 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/526 2024-02-18T16:50:16.081 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:16.084 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:16.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:16.111 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-02-18T16:50:16.346 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:16.350 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:16.351 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:16.352 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:16.352 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:16.658 INFO:teuthology.orchestra.run.smithi116.stdout:generic/527 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T16:50:16.662 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/527 2024-02-18T16:50:16.663 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/527 2024-02-18T16:50:16.663 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:16.665 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:16.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:16.694 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-02-18T16:50:16.926 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:16.931 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:16.931 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:16.932 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:16.932 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:17.334 INFO:teuthology.orchestra.run.smithi116.stdout:generic/528 0s 2024-02-18T16:50:17.341 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/528 2024-02-18T16:50:17.341 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:17.343 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:17.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:17.398 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-02-18T16:50:17.962 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:17.966 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:17.966 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:17.967 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:17.967 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:18.528 INFO:teuthology.orchestra.run.smithi116.stdout:generic/529 0s 2024-02-18T16:50:18.535 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/529 2024-02-18T16:50:18.535 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:18.538 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:18.548 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:18.548 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-02-18T16:50:18.783 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:18.788 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:18.789 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:18.790 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:18.790 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:19.190 INFO:teuthology.orchestra.run.smithi116.stdout:generic/530 [not run] ceph does not support shutdown 2024-02-18T16:50:19.203 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/530 2024-02-18T16:50:19.203 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/530 2024-02-18T16:50:19.203 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:19.206 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:19.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:19.225 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-02-18T16:50:19.458 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:19.463 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:19.464 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:19.465 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:19.465 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:19.775 INFO:teuthology.orchestra.run.smithi116.stdout:generic/531 [not run] O_TMPFILE is not supported 2024-02-18T16:50:19.780 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/531 2024-02-18T16:50:19.780 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/531 2024-02-18T16:50:19.780 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:19.785 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:19.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:19.805 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-02-18T16:50:20.034 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:20.038 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:20.038 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:20.039 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:20.039 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:20.390 INFO:teuthology.orchestra.run.smithi116.stdout:generic/532 0s 2024-02-18T16:50:20.397 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/532 2024-02-18T16:50:20.398 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:20.400 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:20.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:20.438 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-02-18T16:50:20.671 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:20.675 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:20.675 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:20.677 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:20.677 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:21.720 INFO:teuthology.orchestra.run.smithi116.stdout:generic/533 1s 2024-02-18T16:50:21.727 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/533 2024-02-18T16:50:21.727 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:21.733 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:21.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:21.754 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-02-18T16:50:22.010 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:22.014 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:22.015 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:22.015 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:22.016 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:22.308 INFO:teuthology.orchestra.run.smithi116.stdout:generic/534 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T16:50:22.312 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/534 2024-02-18T16:50:22.315 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/534 2024-02-18T16:50:22.315 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:22.316 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:22.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:22.336 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-02-18T16:50:22.574 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:22.578 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:22.579 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:22.580 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:22.580 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:22.864 INFO:teuthology.orchestra.run.smithi116.stdout:generic/535 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T16:50:22.868 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/535 2024-02-18T16:50:22.868 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/535 2024-02-18T16:50:22.868 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:22.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:22.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:22.901 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-02-18T16:50:23.131 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:23.136 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:23.136 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:23.137 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:23.137 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:23.506 INFO:teuthology.orchestra.run.smithi116.stdout:generic/536 [not run] ceph does not support shutdown 2024-02-18T16:50:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/536 2024-02-18T16:50:23.522 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/536 2024-02-18T16:50:23.523 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:23.525 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:23.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:23.552 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-02-18T16:50:23.790 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:23.795 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:23.796 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:23.797 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:23.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:24.159 INFO:teuthology.orchestra.run.smithi116.stdout:generic/537 [not run] FSTRIM not supported 2024-02-18T16:50:24.169 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/537 2024-02-18T16:50:24.169 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/537 2024-02-18T16:50:24.169 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:50:24.172 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:24.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:50:24.191 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-02-18T16:50:24.427 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:50:24.431 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:50:24.431 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:50:24.432 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:50:24.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:50:58.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:50:58.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T16:50:58.886 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T16:50:58.882+0000 7f30f177b640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614755) UID: 0 2024-02-18T16:50:58.887 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T16:50:58.882+0000 7fcaeff29640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 614755) UID: 0 2024-02-18T16:50:58.944 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T16:50:58.940+0000 7f2d640d1640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53178) UID: 0 2024-02-18T16:50:58.944 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T16:50:58.940+0000 7fed528fa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53178) UID: 0 2024-02-18T16:50:58.945 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T16:50:58.940+0000 7fe4fc5cc640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53178) UID: 0 2024-02-18T16:50:58.945 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T16:50:58.940+0000 7ff399284640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 53178) UID: 0 2024-02-18T16:54:07.300 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T16:54:07.295+0000 7f9b36a5c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 617162) UID: 0 2024-02-18T16:54:07.300 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T16:54:07.295+0000 7f348c2aa640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 617162) UID: 0 2024-02-18T16:54:07.301 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T16:54:07.295+0000 7f627de19640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 617162) UID: 0 2024-02-18T16:54:07.301 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:54:07.295+0000 7f5c53248640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 617162) UID: 0 2024-02-18T16:58:29.869 INFO:teuthology.orchestra.run.smithi116.stdout:generic/538 485s 2024-02-18T16:58:30.452 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/538 2024-02-18T16:58:30.452 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:30.455 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:30.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:30.706 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-02-18T16:58:34.748 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:34.752 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:34.752 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:34.753 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:34.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:37.246 INFO:teuthology.orchestra.run.smithi116.stdout:generic/539 3s 2024-02-18T16:58:37.252 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/539 2024-02-18T16:58:37.253 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:37.256 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:37.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:37.328 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-02-18T16:58:37.753 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:37.758 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:37.759 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:37.760 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:37.760 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:38.330 INFO:teuthology.orchestra.run.smithi116.stdout:generic/540 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T16:58:38.336 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/540 2024-02-18T16:58:38.336 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/540 2024-02-18T16:58:38.336 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:38.338 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:38.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:38.743 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-02-18T16:58:38.992 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:38.996 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:38.996 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:38.998 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:38.998 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:39.509 INFO:teuthology.orchestra.run.smithi116.stdout:generic/541 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T16:58:39.517 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/541 2024-02-18T16:58:39.517 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/541 2024-02-18T16:58:39.517 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:39.521 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:39.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:39.875 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-02-18T16:58:40.140 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:40.145 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:40.145 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:40.147 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:40.147 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:40.669 INFO:teuthology.orchestra.run.smithi116.stdout:generic/542 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T16:58:40.677 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/542 2024-02-18T16:58:40.677 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/542 2024-02-18T16:58:40.677 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:40.682 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:41.018 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:41.019 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-02-18T16:58:41.298 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:41.302 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:41.303 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:41.304 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:41.304 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:41.810 INFO:teuthology.orchestra.run.smithi116.stdout:generic/543 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T16:58:41.821 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/543 2024-02-18T16:58:41.821 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/543 2024-02-18T16:58:41.821 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:41.824 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:42.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:42.170 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-02-18T16:58:42.469 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:42.473 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:42.474 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:42.475 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:42.475 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:42.958 INFO:teuthology.orchestra.run.smithi116.stdout:generic/544 [not run] Reflink not supported by scratch filesystem type: ceph 2024-02-18T16:58:42.966 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/544 2024-02-18T16:58:42.966 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/544 2024-02-18T16:58:42.966 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:42.969 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:43.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:43.355 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-02-18T16:58:43.620 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:43.625 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:43.625 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:43.627 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:43.627 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:44.069 INFO:teuthology.orchestra.run.smithi116.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-02-18T16:58:44.075 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/545 2024-02-18T16:58:44.075 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/545 2024-02-18T16:58:44.075 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:44.078 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:44.109 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:44.109 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-02-18T16:58:44.366 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:44.371 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:44.372 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:44.373 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:44.373 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:44.833 INFO:teuthology.orchestra.run.smithi116.stdout:generic/546 [not run] xfs_io falloc failed (old kernel/wrong fs?) 2024-02-18T16:58:44.838 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/546 2024-02-18T16:58:44.838 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/546 2024-02-18T16:58:44.838 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:44.841 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:44.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:44.872 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-02-18T16:58:45.138 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:45.142 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:45.142 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:45.144 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:45.144 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:45.476 INFO:teuthology.orchestra.run.smithi116.stdout:generic/547 [not run] require 172.21.15.116:6789:/scratch to be valid block disk 2024-02-18T16:58:45.480 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/547 2024-02-18T16:58:45.480 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/547 2024-02-18T16:58:45.481 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:45.483 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:45.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:45.504 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-02-18T16:58:45.747 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:45.752 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:45.753 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:45.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:46.267 INFO:teuthology.orchestra.run.smithi116.stdout:generic/548 [not run] kernel does not support ceph encryption 2024-02-18T16:58:46.291 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/548 2024-02-18T16:58:46.291 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/548 2024-02-18T16:58:46.291 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:46.294 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:46.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:46.315 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-02-18T16:58:46.567 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:46.572 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:46.572 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:46.574 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:46.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:47.073 INFO:teuthology.orchestra.run.smithi116.stdout:generic/549 [not run] kernel does not support ceph encryption 2024-02-18T16:58:47.096 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/549 2024-02-18T16:58:47.096 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/549 2024-02-18T16:58:47.096 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:47.099 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:47.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:47.119 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-02-18T16:58:47.387 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:47.393 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:47.393 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:47.394 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:47.394 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:47.875 INFO:teuthology.orchestra.run.smithi116.stdout:generic/550 [not run] kernel does not support ceph encryption 2024-02-18T16:58:47.890 INFO:teuthology.orchestra.run.smithi116.stdout:Ran: generic/550 2024-02-18T16:58:47.890 INFO:teuthology.orchestra.run.smithi116.stdout:Not run: generic/550 2024-02-18T16:58:47.890 INFO:teuthology.orchestra.run.smithi116.stdout:Passed all 1 tests 2024-02-18T16:58:47.895 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:47.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-02-18T16:58:47.923 DEBUG:teuthology.orchestra.run.smithi116:> (cd /tmp/tmp.hC7pXm5hUExfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-02-18T16:58:48.176 INFO:teuthology.orchestra.run.smithi116.stdout:FSTYP -- ceph 2024-02-18T16:58:48.180 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM -- Linux/x86_64 smithi116 5.14.0-421.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Feb 13 19:41:24 UTC 2024 2024-02-18T16:58:48.181 INFO:teuthology.orchestra.run.smithi116.stdout:MKFS_OPTIONS -- 172.21.15.116:6789:/scratch 2024-02-18T16:58:48.182 INFO:teuthology.orchestra.run.smithi116.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQAb7tFl2FzOORAA7t/DbHOkV8hJsoeQbAOyIg== -o context=system_u:object_r:root_t:s0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch 2024-02-18T16:58:48.182 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T16:58:52.335 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:58:52.329+0000 7f5c49852640 -1 bdev(0x56535419ce00 /var/lib/ceph/osd/ceph-3/block) _aio_thread got r=-28 ((28) No space left on device) 2024-02-18T16:58:52.631 INFO:tasks.ceph.osd.3.smithi116.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f5c49852640 time 2024-02-18T16:58:52.378552+0000 2024-02-18T16:58:52.631 INFO:tasks.ceph.osd.3.smithi116.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x56535064a5c6] 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr:*** Caught signal (Aborted) ** 2024-02-18T16:58:52.663 INFO:tasks.ceph.osd.3.smithi116.stderr: in thread 7f5c49852640 thread_name:bstore_aio 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:58:52.659+0000 7f5c49852640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f5c49852640 time 2024-02-18T16:58:52.378552+0000 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x56535064a5c6] 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.664 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c56654db0] 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c566a154c] 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: raise() 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: abort() 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x56535064a687] 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.691 INFO:tasks.ceph.osd.3.smithi116.stderr: 7: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.692 INFO:tasks.ceph.osd.3.smithi116.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.692 INFO:tasks.ceph.osd.3.smithi116.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr:2024-02-18T16:58:52.716+0000 7f5c49852640 -1 *** Caught signal (Aborted) ** 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: in thread 7f5c49852640 thread_name:bstore_aio 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c56654db0] 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c566a154c] 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: raise() 2024-02-18T16:58:52.720 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: abort() 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x56535064a687] 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: 7: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-18T16:58:52.721 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.748 INFO:tasks.ceph.osd.3.smithi116.stderr: -9203> 2024-02-18T16:58:52.329+0000 7f5c49852640 -1 bdev(0x56535419ce00 /var/lib/ceph/osd/ceph-3/block) _aio_thread got r=-28 ((28) No space left on device) 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: -32> 2024-02-18T16:58:52.659+0000 7f5c49852640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f5c49852640 time 2024-02-18T16:58:52.378552+0000 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x56535064a5c6] 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.756 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: -1> 2024-02-18T16:58:52.716+0000 7f5c49852640 -1 *** Caught signal (Aborted) ** 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: in thread 7f5c49852640 thread_name:bstore_aio 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c56654db0] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c566a154c] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: raise() 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: abort() 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x56535064a687] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 7: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.757 INFO:tasks.ceph.osd.3.smithi116.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-18T16:58:52.758 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.772 INFO:tasks.ceph.osd.3.smithi116.stderr: -9217> 2024-02-18T16:58:52.329+0000 7f5c49852640 -1 bdev(0x56535419ce00 /var/lib/ceph/osd/ceph-3/block) _aio_thread got r=-28 ((28) No space left on device) 2024-02-18T16:58:52.779 INFO:tasks.ceph.osd.3.smithi116.stderr: -46> 2024-02-18T16:58:52.659+0000 7f5c49852640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f5c49852640 time 2024-02-18T16:58:52.378552+0000 2024-02-18T16:58:52.779 INFO:tasks.ceph.osd.3.smithi116.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1555-g1c920afa/rpm/el9/BUILD/ceph-19.0.0-1555-g1c920afa/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2024-02-18T16:58:52.779 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.779 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.779 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x56535064a5c6] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: -15> 2024-02-18T16:58:52.716+0000 7f5c49852640 -1 *** Caught signal (Aborted) ** 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: in thread 7f5c49852640 thread_name:bstore_aio 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: ceph version 19.0.0-1555-g1c920afa (1c920afa8316ea21877f3186850254947531ea22) squid (dev) 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5c56654db0] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5c566a154c] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 3: raise() 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 4: abort() 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x187) [0x56535064a687] 2024-02-18T16:58:52.780 INFO:tasks.ceph.osd.3.smithi116.stderr: 6: (KernelDevice::_aio_thread()+0x11e0) [0x5653510125e0] 2024-02-18T16:58:52.781 INFO:tasks.ceph.osd.3.smithi116.stderr: 7: ceph-osd(+0xdbb751) [0x565351012751] 2024-02-18T16:58:52.781 INFO:tasks.ceph.osd.3.smithi116.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f5c5669f802] 2024-02-18T16:58:52.781 INFO:tasks.ceph.osd.3.smithi116.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f5c5663f450] 2024-02-18T16:58:52.781 INFO:tasks.ceph.osd.3.smithi116.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-18T16:58:52.781 INFO:tasks.ceph.osd.3.smithi116.stderr: 2024-02-18T16:58:52.903 INFO:tasks.ceph.osd.3.smithi116.stderr:daemon-helper: command failed with exit status 5 2024-02-18T16:58:55.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-02-18T16:59:01.971 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~7s 2024-02-18T16:59:08.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~13s 2024-02-18T16:59:15.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2024-02-18T16:59:21.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~26s 2024-02-18T16:59:28.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~33s 2024-02-18T16:59:35.040 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2024-02-18T16:59:41.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~46s 2024-02-18T16:59:48.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~53s 2024-02-18T16:59:54.880 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2024-02-18T17:00:01.492 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~66s 2024-02-18T17:00:08.106 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~73s 2024-02-18T17:00:14.719 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~79s 2024-02-18T17:00:21.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~86s 2024-02-18T17:00:27.944 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~93s 2024-02-18T17:00:34.558 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~99s 2024-02-18T17:00:41.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~106s 2024-02-18T17:00:47.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~112s 2024-02-18T17:00:54.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~119s 2024-02-18T17:01:01.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~126s 2024-02-18T17:01:07.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~132s 2024-02-18T17:01:14.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~139s 2024-02-18T17:01:20.855 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~145s 2024-02-18T17:01:27.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~152s 2024-02-18T17:01:34.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~159s 2024-02-18T17:01:40.697 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~165s 2024-02-18T17:01:47.310 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~172s 2024-02-18T17:01:53.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~179s 2024-02-18T17:02:00.537 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~185s 2024-02-18T17:02:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~192s 2024-02-18T17:02:13.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~198s 2024-02-18T17:02:20.373 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~205s 2024-02-18T17:02:26.987 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~212s 2024-02-18T17:02:33.600 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~218s 2024-02-18T17:02:40.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~225s 2024-02-18T17:02:46.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~231s 2024-02-18T17:02:53.440 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~238s 2024-02-18T17:03:00.054 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~245s 2024-02-18T17:03:06.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~251s 2024-02-18T17:03:13.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~258s 2024-02-18T17:03:19.896 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2024-02-18T17:03:26.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~271s 2024-02-18T17:03:33.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~278s 2024-02-18T17:03:39.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~284s 2024-02-18T17:03:46.348 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~291s 2024-02-18T17:03:52.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~298s 2024-02-18T17:03:59.575 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~304s 2024-02-18T17:03:59.575 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-02-18T17:03:59.576 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T17:03:59.576 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T17:04:00.059 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T17:04:00.059 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T17:04:00.085 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-18T17:04:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.682+0000 7ff0068df640 1 Processor -- start 2024-02-18T17:04:02.686 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.682+0000 7ff0068df640 1 -- start start 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7ff0068df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff0000b5ee0 0x7ff000168b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7ff0068df640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0000d3750 con 0x7ff00015ddc0 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7ff0068df640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00015e1a0 con 0x7ff000161f50 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7ff0068df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b96a0 con 0x7ff0000b5ee0 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7feffffff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff00015ddc0 0x7ff000161840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36690/0 (socket says 172.21.15.116:36690) 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7feffffff640 1 -- 172.21.15.116:0/3905351413 learned_addr learned my addr 172.21.15.116:0/3905351413 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7ff004e7c640 1 --2- 172.21.15.116:0/3905351413 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff0000b5ee0 0x7ff000168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:02.687 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.683+0000 7ff004e7c640 1 -- 172.21.15.116:0/3905351413 >> v1:172.21.15.143:6789/0 conn(0x7ff000161f50 legacy=0x7ff0000b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T17:04:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.688+0000 7ff004e7c640 1 -- 172.21.15.116:0/3905351413 >> v1:172.21.15.116:6789/0 conn(0x7ff00015ddc0 legacy=0x7ff000161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:02.692 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.688+0000 7ff004e7c640 1 -- 172.21.15.116:0/3905351413 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0001623e0 con 0x7ff0000b5ee0 2024-02-18T17:04:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.689+0000 7ff004e7c640 1 --2- 172.21.15.116:0/3905351413 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff0000b5ee0 0x7ff000168b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feff8009dc0 tx=0x7feff802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859b0f08f7864831 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.690+0000 7feffeffd640 1 -- 172.21.15.116:0/3905351413 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff802ccd0 con 0x7ff0000b5ee0 2024-02-18T17:04:02.693 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.690+0000 7feffeffd640 1 -- 172.21.15.116:0/3905351413 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff802ce30 con 0x7ff0000b5ee0 2024-02-18T17:04:02.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.690+0000 7feffeffd640 1 -- 172.21.15.116:0/3905351413 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8034660 con 0x7ff0000b5ee0 2024-02-18T17:04:02.694 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 -- 172.21.15.116:0/3905351413 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff0000b5ee0 msgr2=0x7ff000168b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 --2- 172.21.15.116:0/3905351413 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff0000b5ee0 0x7ff000168b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feff8009dc0 tx=0x7feff802bf20 comp rx=0 tx=0).stop 2024-02-18T17:04:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 -- 172.21.15.116:0/3905351413 shutdown_connections 2024-02-18T17:04:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 --2- 172.21.15.116:0/3905351413 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff0000b5ee0 0x7ff000168b70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 -- 172.21.15.116:0/3905351413 >> 172.21.15.116:0/3905351413 conn(0x7ff0000d0870 msgr2=0x7ff0000d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 -- 172.21.15.116:0/3905351413 shutdown_connections 2024-02-18T17:04:02.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.691+0000 7ff0068df640 1 -- 172.21.15.116:0/3905351413 wait complete. 2024-02-18T17:04:02.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.698+0000 7ff0068df640 1 Processor -- start 2024-02-18T17:04:02.702 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 -- start start 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0000b5ee0 0x7ff000152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 0x7ff000150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff00015cde0 0x7ff00015d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000152950 con 0x7ff00015ddc0 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001f3c70 con 0x7ff0000b5ee0 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff0068df640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001f3f50 con 0x7ff00015cde0 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 0x7ff000150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:02.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 0x7ff000150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33822/0 (socket says 172.21.15.116:33822) 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7fefff7fe640 1 -- 172.21.15.116:0/844035665 learned_addr learned my addr 172.21.15.116:0/844035665 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7feffffff640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0000b5ee0 0x7ff000152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.699+0000 7ff004e7c640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff00015cde0 0x7ff00015d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7fefff7fe640 1 -- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff00015cde0 msgr2=0x7ff00015d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7fefff7fe640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff00015cde0 0x7ff00015d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7fefff7fe640 1 -- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0000b5ee0 msgr2=0x7ff000152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7fefff7fe640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0000b5ee0 0x7ff000152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7fefff7fe640 1 -- 172.21.15.116:0/844035665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009a70 con 0x7ff00015ddc0 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7fefff7fe640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 0x7ff000150770 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7feff400c3c0 tx=0x7feff400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40052d0 con 0x7ff00015ddc0 2024-02-18T17:04:02.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0001f4230 con 0x7ff00015ddc0 2024-02-18T17:04:02.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.700+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff000150cb0 con 0x7ff00015ddc0 2024-02-18T17:04:02.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.701+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0000cad50 con 0x7ff00015ddc0 2024-02-18T17:04:02.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.738+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff40010c0 con 0x7ff00015ddc0 2024-02-18T17:04:02.742 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.738+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40035f0 con 0x7ff00015ddc0 2024-02-18T17:04:02.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.795+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7feff4003750 con 0x7ff00015ddc0 2024-02-18T17:04:02.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.796+0000 7feffcff9640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fefec06bca0 0x7fefec06e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:02.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.796+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7feff408a3e0 con 0x7ff00015ddc0 2024-02-18T17:04:02.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.796+0000 7feffffff640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fefec06bca0 0x7fefec06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:02.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.796+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feff408a860 con 0x7ff00015ddc0 2024-02-18T17:04:02.800 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:02.796+0000 7feffffff640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fefec06bca0 0x7fefec06e160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff0000ca8b0 tx=0x7feff0007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.011 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.008+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff00015e1a0 con 0x7ff00015ddc0 2024-02-18T17:04:03.012 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.009+0000 7feffcff9640 1 -- 172.21.15.116:0/844035665 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17088 (secure 0 0 0) 0x7ff00015e1a0 con 0x7ff00015ddc0 2024-02-18T17:04:03.013 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-02-18T17:04:03.013 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":42,"fsid":"82e3e917-bb19-48fa-a66a-62df77a0f413","created":"2024-02-18T11:47:10.104830+0000","modified":"2024-02-18T16:58:54.323472+0000","last_up_change":"2024-02-18T16:58:52.998534+0000","last_in_change":"2024-02-18T11:47:12.335646+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-18T11:47:22.334088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-02-18T11:48:44.953507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-02-18T11:48:45.607007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7af0914b-25ef-478f-812e-7e545d4216b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6801","nonce":991283493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6803","nonce":991283493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6807","nonce":991283493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":991283493},{"type":"v1","addr":"172.21.15.116:6805","nonce":991283493}]},"public_addr":"172.21.15.116:6801/991283493","cluster_addr":"172.21.15.116:6803/991283493","heartbeat_back_addr":"172.21.15.116:6807/991283493","heartbeat_front_addr":"172.21.15.116:6805/991283493","state":["exists","up"]},{"osd":1,"uuid":"f78069a3-e4be-4438-9956-85482d8671c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6809","nonce":208468530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6811","nonce":208468530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6815","nonce":208468530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":208468530},{"type":"v1","addr":"172.21.15.116:6813","nonce":208468530}]},"public_addr":"172.21.15.116:6809/208468530","cluster_addr":"172.21.15.116:6811/208468530","heartbeat_back_addr":"172.21.15.116:6815/208468530","heartbeat_front_addr":"172.21.15.116:6813/208468530","state":["exists","up"]},{"osd":2,"uuid":"391fd049-5adf-469c-a031-e1d2cf36f723","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6817","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6819","nonce":3065364614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6821","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6823","nonce":3065364614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6829","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6831","nonce":3065364614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6825","nonce":3065364614},{"type":"v1","addr":"172.21.15.116:6827","nonce":3065364614}]},"public_addr":"172.21.15.116:6819/3065364614","cluster_addr":"172.21.15.116:6823/3065364614","heartbeat_back_addr":"172.21.15.116:6831/3065364614","heartbeat_front_addr":"172.21.15.116:6827/3065364614","state":["exists","up"]},{"osd":3,"uuid":"755e4fc9-1965-4e13-9955-68bc4ff41573","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":41,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6818","nonce":1865949701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6822","nonce":1865949701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6830","nonce":1865949701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":1865949701},{"type":"v1","addr":"172.21.15.116:6826","nonce":1865949701}]},"public_addr":"172.21.15.116:6818/1865949701","cluster_addr":"172.21.15.116:6822/1865949701","heartbeat_back_addr":"172.21.15.116:6830/1865949701","heartbeat_front_addr":"172.21.15.116:6826/1865949701","state":["exists"]},{"osd":4,"uuid":"429ccd4b-5332-496c-b884-7d88777a186d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6803","nonce":2574593080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6808","nonce":2574593080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6815","nonce":2574593080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2574593080},{"type":"v1","addr":"172.21.15.143:6813","nonce":2574593080}]},"public_addr":"172.21.15.143:6803/2574593080","cluster_addr":"172.21.15.143:6808/2574593080","heartbeat_back_addr":"172.21.15.143:6815/2574593080","heartbeat_front_addr":"172.21.15.143:6813/2574593080","state":["exists","up"]},{"osd":5,"uuid":"25dce560-148e-49a8-a32b-3bcb4b6bd446","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6825","nonce":680136748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6827","nonce":680136748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6831","nonce":680136748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":680136748},{"type":"v1","addr":"172.21.15.143:6829","nonce":680136748}]},"public_addr":"172.21.15.143:6825/680136748","cluster_addr":"172.21.15.143:6827/680136748","heartbeat_back_addr":"172.21.15.143:6831/680136748","heartbeat_front_addr":"172.21.15.143:6829/680136748","state":["exists","up"]},{"osd":6,"uuid":"2239dfcb-8d00-4ce1-8f4f-9d366513bafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6801","nonce":3882163910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6805","nonce":3882163910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6812","nonce":3882163910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":3882163910},{"type":"v1","addr":"172.21.15.143:6809","nonce":3882163910}]},"public_addr":"172.21.15.143:6801/3882163910","cluster_addr":"172.21.15.143:6805/3882163910","heartbeat_back_addr":"172.21.15.143:6812/3882163910","heartbeat_front_addr":"172.21.15.143:6809/3882163910","state":["exists","up"]},{"osd":7,"uuid":"ab909074-4daa-42df-b320-e78bf4f1e2dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6817","nonce":374457617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6819","nonce":374457617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6823","nonce":374457617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":374457617},{"type":"v1","addr":"172.21.15.143:6821","nonce":374457617}]},"public_addr":"172.21.15.143:6817/374457617","cluster_addr":"172.21.15.143:6819/374457617","heartbeat_back_addr":"172.21.15.143:6823/374457617","heartbeat_front_addr":"172.21.15.143:6821/374457617","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.485518+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.573399+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.508532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.500311+0000","dead_epoch":41},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.525868+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.534690+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.434158+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T11:47:17.523700+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fefec06bca0 msgr2=0x7fefec06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fefec06bca0 0x7fefec06e160 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff0000ca8b0 tx=0x7feff0007e80 comp rx=0 tx=0).stop 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 msgr2=0x7ff000150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 0x7ff000150770 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7feff400c3c0 tx=0x7feff400c880 comp rx=0 tx=0).stop 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 shutdown_connections 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fefec06bca0 0x7fefec06e160 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff00015cde0 0x7ff00015d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff00015ddc0 0x7ff000150770 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 --2- 172.21.15.116:0/844035665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0000b5ee0 0x7ff000152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 >> 172.21.15.116:0/844035665 conn(0x7ff0000d0870 msgr2=0x7ff00015a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 shutdown_connections 2024-02-18T17:04:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.011+0000 7ff0068df640 1 -- 172.21.15.116:0/844035665 wait complete. 2024-02-18T17:04:03.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-18T11:47:22.334088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-18T11:48:44.953507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-02-18T11:48:45.607007+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.4900000095367432, 'score_stable': 1.4900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-18T17:04:03.031 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-18T17:04:03.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.145+0000 7f1e58a4e640 1 Processor -- start 2024-02-18T17:04:03.149 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.145+0000 7f1e58a4e640 1 -- start start 2024-02-18T17:04:03.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e58a4e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e58a4e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e540b93b0 0x7f1e540ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e58a4e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e540cf180 con 0x7f1e540ced30 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e58a4e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e540d21b0 con 0x7f1e540b93b0 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e58a4e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540d1e60 con 0x7f1e5415dba0 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e52d9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1e540ced30 0x7f1e5406c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36704/0 (socket says 172.21.15.116:36704) 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e52d9d640 1 -- 172.21.15.116:0/1912039554 learned_addr learned my addr 172.21.15.116:0/1912039554 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e49d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e540b93b0 0x7f1e540ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e5259c640 1 --2- 172.21.15.116:0/1912039554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e51d9b640 1 -- 172.21.15.116:0/1912039554 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311699952 0 0) 0x7f1e4c001490 con 0x7f1e540ced30 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.146+0000 7f1e51d9b640 1 -- 172.21.15.116:0/1912039554 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e40003520 con 0x7f1e540ced30 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.147+0000 7f1e49d9b640 1 -- 172.21.15.116:0/1912039554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 msgr2=0x7f1e54161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.147+0000 7f1e49d9b640 1 --2- 172.21.15.116:0/1912039554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.147+0000 7f1e49d9b640 1 -- 172.21.15.116:0/1912039554 >> v1:172.21.15.116:6789/0 conn(0x7f1e540ced30 legacy=0x7f1e5406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.147+0000 7f1e49d9b640 1 -- 172.21.15.116:0/1912039554 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e54162050 con 0x7f1e540b93b0 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.148+0000 7f1e49d9b640 1 --2- 172.21.15.116:0/1912039554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540b93b0 0x7f1e540ce7f0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0097a0 tx=0x7f1e3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95583a9951b3aea server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.148+0000 7f1e51d9b640 1 -- 172.21.15.116:0/1912039554 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c031830 con 0x7f1e540b93b0 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.148+0000 7f1e51d9b640 1 -- 172.21.15.116:0/1912039554 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c031990 con 0x7f1e540b93b0 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.148+0000 7f1e51d9b640 1 -- 172.21.15.116:0/1912039554 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c031d20 con 0x7f1e540b93b0 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 -- 172.21.15.116:0/1912039554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540b93b0 msgr2=0x7f1e540ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/1912039554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540b93b0 0x7f1e540ce7f0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c0097a0 tx=0x7f1e3c02b8d0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 -- 172.21.15.116:0/1912039554 shutdown_connections 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/1912039554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540b93b0 0x7f1e540ce7f0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/1912039554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 -- 172.21.15.116:0/1912039554 >> 172.21.15.116:0/1912039554 conn(0x7f1e540d04a0 msgr2=0x7f1e540d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:03.153 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 -- 172.21.15.116:0/1912039554 shutdown_connections 2024-02-18T17:04:03.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.149+0000 7f1e58a4e640 1 -- 172.21.15.116:0/1912039554 wait complete. 2024-02-18T17:04:03.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.151+0000 7f1e58a4e640 1 Processor -- start 2024-02-18T17:04:03.154 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.151+0000 7f1e58a4e640 1 -- start start 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e58a4e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 0x7f1e5415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e58a4e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540ced30 0x7f1e540c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e58a4e640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e58a4e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c8f60 con 0x7f1e540ced30 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e58a4e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c9240 con 0x7f1e540b93b0 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e58a4e640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e540c9520 con 0x7f1e5415dba0 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 0x7f1e5415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e52d9d640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.157 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 0x7f1e5415e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.116:54364/0 (socket says 172.21.15.116:54364) 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 -- 172.21.15.116:0/3565237013 learned_addr learned my addr 172.21.15.116:0/3565237013 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 -- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 msgr2=0x7f1e54147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 -- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540ced30 msgr2=0x7f1e540c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e49d9b640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540ced30 0x7f1e540c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540ced30 0x7f1e540c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e5259c640 1 -- 172.21.15.116:0/3565237013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c009400 con 0x7f1e540b93b0 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.152+0000 7f1e49d9b640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540ced30 0x7f1e540c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.153+0000 7f1e5259c640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 0x7f1e5415e1f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400e870 tx=0x7f1e4400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.153+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e440094f0 con 0x7f1e540b93b0 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.153+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e540c9800 con 0x7f1e540b93b0 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.153+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5415e730 con 0x7f1e540b93b0 2024-02-18T17:04:03.158 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.153+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4400c940 con 0x7f1e540b93b0 2024-02-18T17:04:03.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.153+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400ccf0 con 0x7f1e540b93b0 2024-02-18T17:04:03.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.154+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e38005290 con 0x7f1e540b93b0 2024-02-18T17:04:03.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.155+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f1e44013470 con 0x7f1e540b93b0 2024-02-18T17:04:03.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.155+0000 7f1e4b7fe640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1e3406bc70 0x7f1e3406e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.156+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f1e4408a940 con 0x7f1e540b93b0 2024-02-18T17:04:03.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.156+0000 7f1e49d9b640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1e3406bc70 0x7f1e3406e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.156+0000 7f1e49d9b640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1e3406bc70 0x7f1e3406e130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c036000 tx=0x7f1e3c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.162+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e44055020 con 0x7f1e540b93b0 2024-02-18T17:04:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.450+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e38005810 con 0x7f1e540b93b0 2024-02-18T17:04:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.451+0000 7f1e4b7fe640 1 -- 172.21.15.116:0/3565237013 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e44017020 con 0x7f1e540b93b0 2024-02-18T17:04:03.455 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.454+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1e3406bc70 msgr2=0x7f1e3406e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.454+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1e3406bc70 0x7f1e3406e130 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c036000 tx=0x7f1e3c008030 comp rx=0 tx=0).stop 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 msgr2=0x7f1e5415e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 0x7f1e5415e1f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400e870 tx=0x7f1e4400ed30 comp rx=0 tx=0).stop 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 shutdown_connections 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f1e3406bc70 0x7f1e3406e130 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1e5415dba0 0x7f1e54147050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e540ced30 0x7f1e540c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 --2- 172.21.15.116:0/3565237013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1e540b93b0 0x7f1e5415e1f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 >> 172.21.15.116:0/3565237013 conn(0x7f1e540d04a0 msgr2=0x7f1e540ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 shutdown_connections 2024-02-18T17:04:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.455+0000 7f1e58a4e640 1 -- 172.21.15.116:0/3565237013 wait complete. 2024-02-18T17:04:03.479 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-18T17:04:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.640+0000 7fa83d995640 1 Processor -- start 2024-02-18T17:04:03.644 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.640+0000 7fa83d995640 1 -- start start 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.641+0000 7fa83d995640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8300df6f0 0x7fa830054cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.641+0000 7fa83d995640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa83003ad50 0x7fa830053400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.641+0000 7fa83d995640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa830053c30 con 0x7fa830050090 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.641+0000 7fa83d995640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300dfad0 con 0x7fa8300df6f0 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.641+0000 7fa83d995640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa830053f10 con 0x7fa83003ad50 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa83c993640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8300df6f0 0x7fa830054cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa83c993640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8300df6f0 0x7fa830054cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33846/0 (socket says 172.21.15.116:33846) 2024-02-18T17:04:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa83c993640 1 -- 172.21.15.116:0/3020498494 learned_addr learned my addr 172.21.15.116:0/3020498494 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T17:04:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa837fff640 1 --2- 172.21.15.116:0/3020498494 >> v2:172.21.15.143:3300/0 conn(0x7fa83003ad50 0x7fa830053400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa837fff640 1 -- 172.21.15.116:0/3020498494 >> v2:172.21.15.116:3300/0 conn(0x7fa8300df6f0 msgr2=0x7fa830054cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T17:04:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa837fff640 1 --2- 172.21.15.116:0/3020498494 >> v2:172.21.15.116:3300/0 conn(0x7fa8300df6f0 0x7fa830054cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa837fff640 1 -- 172.21.15.116:0/3020498494 >> v1:172.21.15.143:6789/0 conn(0x7fa830050090 legacy=0x7fa8300504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.642+0000 7fa837fff640 1 -- 172.21.15.116:0/3020498494 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300541f0 con 0x7fa83003ad50 2024-02-18T17:04:03.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.643+0000 7fa837fff640 1 --2- 172.21.15.116:0/3020498494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa830053400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa83000e7d0 tx=0x7fa82402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e91623efb4e6d48 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.644+0000 7fa8377fe640 1 -- 172.21.15.116:0/3020498494 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82402e070 con 0x7fa83003ad50 2024-02-18T17:04:03.648 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.644+0000 7fa8377fe640 1 -- 172.21.15.116:0/3020498494 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8240315a0 con 0x7fa83003ad50 2024-02-18T17:04:03.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.644+0000 7fa8377fe640 1 -- 172.21.15.116:0/3020498494 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824031930 con 0x7fa83003ad50 2024-02-18T17:04:03.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.645+0000 7fa83d995640 1 -- 172.21.15.116:0/3020498494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 msgr2=0x7fa830053400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.645+0000 7fa83d995640 1 --2- 172.21.15.116:0/3020498494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa830053400 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa83000e7d0 tx=0x7fa82402b840 comp rx=0 tx=0).stop 2024-02-18T17:04:03.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.645+0000 7fa83d995640 1 -- 172.21.15.116:0/3020498494 shutdown_connections 2024-02-18T17:04:03.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.645+0000 7fa83d995640 1 --2- 172.21.15.116:0/3020498494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa830053400 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.645+0000 7fa83d995640 1 --2- 172.21.15.116:0/3020498494 >> v2:172.21.15.116:3300/0 conn(0x7fa8300df6f0 0x7fa830054cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.645+0000 7fa83d995640 1 -- 172.21.15.116:0/3020498494 >> 172.21.15.116:0/3020498494 conn(0x7fa830055350 msgr2=0x7fa830055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.646+0000 7fa83d995640 1 -- 172.21.15.116:0/3020498494 shutdown_connections 2024-02-18T17:04:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.646+0000 7fa83d995640 1 -- 172.21.15.116:0/3020498494 wait complete. 2024-02-18T17:04:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.648+0000 7fa83d995640 1 Processor -- start 2024-02-18T17:04:03.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.648+0000 7fa83d995640 1 -- start start 2024-02-18T17:04:03.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d995640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa8300d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d995640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 0x7fa83004c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d995640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8300df6f0 0x7fa83017a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d995640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300d1d10 con 0x7fa830050090 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d995640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300dfda0 con 0x7fa83003ad50 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d995640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8300e24e0 con 0x7fa8300df6f0 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa837fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 0x7fa83004c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa837fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 0x7fa83004c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33848/0 (socket says 172.21.15.116:33848) 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa837fff640 1 -- 172.21.15.116:0/461388379 learned_addr learned my addr 172.21.15.116:0/461388379 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83d194640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8300df6f0 0x7fa83017a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.649+0000 7fa83c993640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa8300d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa837fff640 1 -- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8300df6f0 msgr2=0x7fa83017a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa837fff640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8300df6f0 0x7fa83017a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa837fff640 1 -- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 msgr2=0x7fa8300d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:03.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa837fff640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa8300d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa837fff640 1 -- 172.21.15.116:0/461388379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009400 con 0x7fa830050090 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa837fff640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 0x7fa83004c420 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fa824009890 tx=0x7fa8240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824032840 con 0x7fa830050090 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8240329a0 con 0x7fa830050090 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8300e27c0 con 0x7fa830050090 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824032d50 con 0x7fa830050090 2024-02-18T17:04:03.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.650+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa830036460 con 0x7fa830050090 2024-02-18T17:04:03.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.652+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8300e2ce0 con 0x7fa830050090 2024-02-18T17:04:03.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.652+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fa82404d3f0 con 0x7fa830050090 2024-02-18T17:04:03.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.653+0000 7fa8357fa640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa81c06bc70 0x7fa81c06e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:03.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.653+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fa8240ac300 con 0x7fa830050090 2024-02-18T17:04:03.657 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.653+0000 7fa83c993640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa81c06bc70 0x7fa81c06e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:03.658 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.654+0000 7fa83c993640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa81c06bc70 0x7fa81c06e130 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8300de430 tx=0x7fa82c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:03.668 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.665+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8300e2ce0 con 0x7fa830050090 2024-02-18T17:04:03.938 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.934+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8300e2ce0 con 0x7fa830050090 2024-02-18T17:04:04.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.934+0000 7fa8357fa640 1 -- 172.21.15.116:0/461388379 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8300e2ce0 con 0x7fa830050090 2024-02-18T17:04:04.024 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.937+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa81c06bc70 msgr2=0x7fa81c06e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.937+0000 7fa83d995640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa81c06bc70 0x7fa81c06e130 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8300de430 tx=0x7fa82c009240 comp rx=0 tx=0).stop 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.937+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 msgr2=0x7fa83004c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.937+0000 7fa83d995640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 0x7fa83004c420 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fa824009890 tx=0x7fa8240046c0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 shutdown_connections 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fa81c06bc70 0x7fa81c06e130 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa8300df6f0 0x7fa83017a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa830050090 0x7fa83004c420 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 --2- 172.21.15.116:0/461388379 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa83003ad50 0x7fa8300d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 >> 172.21.15.116:0/461388379 conn(0x7fa830055350 msgr2=0x7fa83003a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 shutdown_connections 2024-02-18T17:04:04.026 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:03.938+0000 7fa83d995640 1 -- 172.21.15.116:0/461388379 wait complete. 2024-02-18T17:04:04.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-18T17:04:04.159 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.156+0000 7fca4117a640 1 Processor -- start 2024-02-18T17:04:04.160 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.156+0000 7fca4117a640 1 -- start start 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca4117a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c0cef00 0x7fca3c168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca4117a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0d21b0 con 0x7fca3c15dba0 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca4117a640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0d1e60 con 0x7fca3c0b93b0 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca4117a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c162050 con 0x7fca3c0cef00 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca3ad9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fca3c15dba0 0x7fca3c161800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36706/0 (socket says 172.21.15.116:36706) 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca3ad9d640 1 -- 172.21.15.116:0/2808048440 learned_addr learned my addr 172.21.15.116:0/2808048440 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca3b59e640 1 --2- 172.21.15.116:0/2808048440 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c0cef00 0x7fca3c168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.161 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca39d9b640 1 -- 172.21.15.116:0/2808048440 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901646908 0 0) 0x7fca3c0d21b0 con 0x7fca3c15dba0 2024-02-18T17:04:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca39d9b640 1 -- 172.21.15.116:0/2808048440 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca24003520 con 0x7fca3c15dba0 2024-02-18T17:04:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca39d9b640 1 -- 172.21.15.116:0/2808048440 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037395465 0 0) 0x7fca24003520 con 0x7fca3c15dba0 2024-02-18T17:04:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca3b59e640 1 -- 172.21.15.116:0/2808048440 >> v1:172.21.15.143:6789/0 conn(0x7fca3c0b93b0 legacy=0x7fca3c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca3b59e640 1 -- 172.21.15.116:0/2808048440 >> v1:172.21.15.116:6789/0 conn(0x7fca3c15dba0 legacy=0x7fca3c161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.157+0000 7fca3b59e640 1 -- 172.21.15.116:0/2808048440 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c162330 con 0x7fca3c0cef00 2024-02-18T17:04:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.158+0000 7fca3b59e640 1 --2- 172.21.15.116:0/2808048440 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c0cef00 0x7fca3c168b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca34009dc0 tx=0x7fca3402bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa3c9751d6f68685 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:04.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.196+0000 7fca39d9b640 1 -- 172.21.15.116:0/2808048440 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c001670 con 0x7fca3c0cef00 2024-02-18T17:04:04.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.196+0000 7fca39d9b640 1 -- 172.21.15.116:0/2808048440 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3402c910 con 0x7fca3c0cef00 2024-02-18T17:04:04.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.196+0000 7fca39d9b640 1 -- 172.21.15.116:0/2808048440 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3402cca0 con 0x7fca3c0cef00 2024-02-18T17:04:04.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 -- 172.21.15.116:0/2808048440 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c0cef00 msgr2=0x7fca3c168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.202 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 --2- 172.21.15.116:0/2808048440 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c0cef00 0x7fca3c168b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca34009dc0 tx=0x7fca3402bde0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 -- 172.21.15.116:0/2808048440 shutdown_connections 2024-02-18T17:04:04.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 --2- 172.21.15.116:0/2808048440 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c0cef00 0x7fca3c168b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 -- 172.21.15.116:0/2808048440 >> 172.21.15.116:0/2808048440 conn(0x7fca3c0d04a0 msgr2=0x7fca3c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:04.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 -- 172.21.15.116:0/2808048440 shutdown_connections 2024-02-18T17:04:04.203 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.197+0000 7fca4117a640 1 -- 172.21.15.116:0/2808048440 wait complete. 2024-02-18T17:04:04.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.199+0000 7fca4117a640 1 Processor -- start 2024-02-18T17:04:04.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.200+0000 7fca4117a640 1 -- start start 2024-02-18T17:04:04.207 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca4117a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca3c0b93b0 0x7fca3c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca4117a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 0x7fca3c0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca4117a640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c15dba0 0x7fca3c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca4117a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0cb9b0 con 0x7fca3c0cef00 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca4117a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c1f6c50 con 0x7fca3c0b93b0 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca4117a640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c1f6ea0 con 0x7fca3c15dba0 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca3a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 0x7fca3c0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca3a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 0x7fca3c0c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33864/0 (socket says 172.21.15.116:33864) 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca3a59c640 1 -- 172.21.15.116:0/3627974354 learned_addr learned my addr 172.21.15.116:0/3627974354 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca3b59e640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c15dba0 0x7fca3c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.208 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.201+0000 7fca3ad9d640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca3c0b93b0 0x7fca3c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca3a59c640 1 -- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c15dba0 msgr2=0x7fca3c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca3a59c640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c15dba0 0x7fca3c0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca3a59c640 1 -- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca3c0b93b0 msgr2=0x7fca3c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca3a59c640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca3c0b93b0 0x7fca3c0cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca3a59c640 1 -- 172.21.15.116:0/3627974354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34009a70 con 0x7fca3c0cef00 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca3a59c640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 0x7fca3c0c8b20 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fca3000c400 tx=0x7fca3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca300052d0 con 0x7fca3c0cef00 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca30005430 con 0x7fca3c0cef00 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c1f7180 con 0x7fca3c0cef00 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30003830 con 0x7fca3c0cef00 2024-02-18T17:04:04.209 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.202+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c1f7740 con 0x7fca3c0cef00 2024-02-18T17:04:04.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.206+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7fca30003990 con 0x7fca3c0cef00 2024-02-18T17:04:04.210 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.206+0000 7fca237fe640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fca2806bc70 0x7fca2806e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.206+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fca30089e20 con 0x7fca3c0cef00 2024-02-18T17:04:04.211 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.207+0000 7fca3ad9d640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fca2806bc70 0x7fca2806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.212 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.209+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c0cf950 con 0x7fca3c0cef00 2024-02-18T17:04:04.214 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.211+0000 7fca3ad9d640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fca2806bc70 0x7fca2806e130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005880 tx=0x7fca2c001da0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:04.221 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.218+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca300544f0 con 0x7fca3c0cef00 2024-02-18T17:04:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.498+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca3c0d1730 con 0x7fca3c0cef00 2024-02-18T17:04:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.502+0000 7fca237fe640 1 -- 172.21.15.116:0/3627974354 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca30017020 con 0x7fca3c0cef00 2024-02-18T17:04:04.505 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 64 2024-02-18T17:04:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fca2806bc70 msgr2=0x7fca2806e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fca2806bc70 0x7fca2806e130 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fca2c005880 tx=0x7fca2c001da0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 msgr2=0x7fca3c0c8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 0x7fca3c0c8b20 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fca3000c400 tx=0x7fca3000c8c0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 shutdown_connections 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7fca2806bc70 0x7fca2806e130 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca3c15dba0 0x7fca3c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca3c0cef00 0x7fca3c0c8b20 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.506+0000 7fca4117a640 1 --2- 172.21.15.116:0/3627974354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca3c0b93b0 0x7fca3c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.507+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 >> 172.21.15.116:0/3627974354 conn(0x7fca3c0d04a0 msgr2=0x7fca3c0d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.507+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 shutdown_connections 2024-02-18T17:04:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.507+0000 7fca4117a640 1 -- 172.21.15.116:0/3627974354 wait complete. 2024-02-18T17:04:04.533 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-18T17:04:04.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.675+0000 7f76b6d56640 1 Processor -- start 2024-02-18T17:04:04.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.675+0000 7f76b6d56640 1 -- start start 2024-02-18T17:04:04.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76b6d56640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76b0071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76b6d56640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b016b750 con 0x7f76b00cde00 2024-02-18T17:04:04.695 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76b6d56640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b016ba30 con 0x7f76b016a830 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76b6d56640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0071fa0 con 0x7f76b00715e0 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76b52f3640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f76b00cde00 0x7f76b00ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:36710/0 (socket says 172.21.15.116:36710) 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76b52f3640 1 -- 172.21.15.116:0/3250176189 learned_addr learned my addr 172.21.15.116:0/3250176189 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76affff640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76b0071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.676+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672981508 0 0) 0x7f76b016b750 con 0x7f76b00cde00 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.690+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a0003db0 con 0x7f76b00cde00 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.690+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.1 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383786086 0 0) 0x7f76b016ba30 con 0x7f76b016a830 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.690+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b016b750 con 0x7f76b016a830 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1607761633 0 0) 0x7f76a0003db0 con 0x7f76b00cde00 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 msgr2=0x7f76b0071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76b0071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 >> v1:172.21.15.143:6789/0 conn(0x7f76b016a830 legacy=0x7f76b016ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.696 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b00cd600 con 0x7f76b00cde00 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1731835876 0 0) 0x7f76a80019f0 con 0x7f76b00cde00 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 >> v1:172.21.15.116:6789/0 conn(0x7f76b00cde00 legacy=0x7f76b00ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76a0003420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a00043c0 0x7f76a0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a0005050 0x7f76a00054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0071fa0 con 0x7f76a00043c0 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0005a10 con 0x7f76a0005050 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0005bd0 con 0x7f76b00715e0 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76b4af2640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76a0003420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76affff640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a00043c0 0x7f76a0004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.697 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.691+0000 7f76b52f3640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a0005050 0x7f76a00054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.692+0000 7f76b4af2640 1 -- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a0005050 msgr2=0x7f76a00054d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T17:04:04.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.692+0000 7f76b4af2640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76a0005050 0x7f76a00054d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.692+0000 7f76b4af2640 1 -- 172.21.15.116:0/3250176189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a00043c0 msgr2=0x7f76a0004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-18T17:04:04.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.692+0000 7f76b4af2640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a00043c0 0x7f76a0004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.698 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.692+0000 7f76b4af2640 1 -- 172.21.15.116:0/3250176189 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0003db0 con 0x7f76b00715e0 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.693+0000 7f76b4af2640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76a0003420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001940 tx=0x7f76a0006080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f978944ad0c110b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.693+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a40052d0 con 0x7f76b00715e0 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a4005450 con 0x7f76b00715e0 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76af7fe640 1 -- 172.21.15.116:0/3250176189 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a402ebf0 con 0x7f76b00715e0 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76b6d56640 1 -- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 msgr2=0x7f76a0003420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76b6d56640 1 --2- 172.21.15.116:0/3250176189 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b00715e0 0x7f76a0003420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001940 tx=0x7f76a0006080 comp rx=0 tx=0).stop 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76b52f3640 1 -- 172.21.15.116:0/3250176189 reap_dead start 2024-02-18T17:04:04.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76b6d56640 1 -- 172.21.15.116:0/3250176189 shutdown_connections 2024-02-18T17:04:04.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.694+0000 7f76b6d56640 1 -- 172.21.15.116:0/3250176189 >> 172.21.15.116:0/3250176189 conn(0x7f76b006af80 msgr2=0x7f76b006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:04.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.696+0000 7f76b6d56640 1 -- 172.21.15.116:0/3250176189 shutdown_connections 2024-02-18T17:04:04.700 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.696+0000 7f76b6d56640 1 -- 172.21.15.116:0/3250176189 wait complete. 2024-02-18T17:04:04.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.698+0000 7f76b6d56640 1 Processor -- start 2024-02-18T17:04:04.701 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.698+0000 7f76b6d56640 1 -- start start 2024-02-18T17:04:04.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b6d56640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76b00715e0 0x7f76b0165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b6d56640 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b015ae40 0x7f76b016d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b6d56640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 0x7f76b0204e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b6d56640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b016ca70 con 0x7f76b0204980 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b6d56640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b016f120 con 0x7f76b00715e0 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b6d56640 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b016e8c0 con 0x7f76b015ae40 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b52f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 0x7f76b0204e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b52f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 0x7f76b0204e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:33886/0 (socket says 172.21.15.116:33886) 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b4af2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76b00715e0 0x7f76b0165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76b52f3640 1 -- 172.21.15.116:0/4270636319 learned_addr learned my addr 172.21.15.116:0/4270636319 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-02-18T17:04:04.705 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.699+0000 7f76affff640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b015ae40 0x7f76b016d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b52f3640 1 -- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b015ae40 msgr2=0x7f76b016d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b52f3640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b015ae40 0x7f76b016d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b52f3640 1 -- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76b00715e0 msgr2=0x7f76b0165cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b52f3640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76b00715e0 0x7f76b0165cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b52f3640 1 -- 172.21.15.116:0/4270636319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b015b2c0 con 0x7f76b0204980 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b52f3640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 0x7f76b0204e00 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f76b00b4400 tx=0x7f76a800a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a800ac00 con 0x7f76b0204980 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a800ad60 con 0x7f76b0204980 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b016eb60 con 0x7f76b0204980 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80135f0 con 0x7f76b0204980 2024-02-18T17:04:04.706 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.700+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b016cc60 con 0x7f76b0204980 2024-02-18T17:04:04.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.702+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b0074520 con 0x7f76b0204980 2024-02-18T17:04:04.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.702+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 90106+0+0 (secure 0 0 0) 0x7f76a8013750 con 0x7f76b0204980 2024-02-18T17:04:04.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.702+0000 7f76ad7fa640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f769406bca0 0x7f769406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-18T17:04:04.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.702+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f76a808aed0 con 0x7f76b0204980 2024-02-18T17:04:04.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.703+0000 7f76b4af2640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f769406bca0 0x7f769406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-18T17:04:04.707 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.703+0000 7f76b4af2640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f769406bca0 0x7f769406e160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f76a4036000 tx=0x7f76a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-18T17:04:04.714 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:04.710+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76a8055590 con 0x7f76b0204980 2024-02-18T17:04:05.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.016+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f76b00b4430 con 0x7f76b0204980 2024-02-18T17:04:05.020 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.017+0000 7f76ad7fa640 1 -- 172.21.15.116:0/4270636319 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f76a8058c50 con 0x7f76b0204980 2024-02-18T17:04:05.021 INFO:teuthology.orchestra.run.smithi116.stderr:listed 0 entries 2024-02-18T17:04:05.022 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f769406bca0 msgr2=0x7f769406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:05.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f769406bca0 0x7f769406e160 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f76a4036000 tx=0x7f76a4009040 comp rx=0 tx=0).stop 2024-02-18T17:04:05.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 msgr2=0x7f76b0204e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T17:04:05.023 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 0x7f76b0204e00 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7f76b00b4400 tx=0x7f76a800a600 comp rx=0 tx=0).stop 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 shutdown_connections 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:6832/4274496429,v1:172.21.15.116:6833/4274496429] conn(0x7f769406bca0 0x7f769406e160 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b0204980 0x7f76b0204e00 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f76b015ae40 0x7f76b016d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 --2- 172.21.15.116:0/4270636319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76b00715e0 0x7f76b0165cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.019+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 >> 172.21.15.116:0/4270636319 conn(0x7f76b006af80 msgr2=0x7f76b00af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.020+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 shutdown_connections 2024-02-18T17:04:05.024 INFO:teuthology.orchestra.run.smithi116.stderr:2024-02-18T17:04:05.020+0000 7f76b6d56640 1 -- 172.21.15.116:0/4270636319 wait complete. 2024-02-18T17:04:05.040 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-18T17:04:05.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T17:04:05.041 DEBUG:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-02-18T17:04:05.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T17:04:05.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T17:04:05.084 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-18T17:04:05.206 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-02-18T17:04:05.206 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T17:04:05.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056371,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20574 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 172.21.15.116:6789:/test /tmp/tmp.jguMaGqYX4test ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413,acl 0 0 172.21.15.116:6789:/scratch /tmp/tmp.gu5YvuvMAWscratch ceph rw,context=system_u:object_r:root_t:s0,relatime,name=admin,secret=,fsid=82e3e917-bb19-48fa-a66a-62df77a0f413,acl 0 0 2024-02-18T17:04:05.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-02-18T17:04:05.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T17:04:05.223 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-02-18T17:04:05.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T17:04:05.281 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-02-18T17:04:05.281 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T17:04:05.282 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T17:04:06.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-18T17:04:06.229 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T17:04:06.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T17:04:06.229 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-02-18T17:04:06.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T17:04:06.392 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-02-18T17:04:06.392 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-02-18T17:04:06.392 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-18T17:04:06.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056372,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2731 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-02-18T17:04:06.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-02-18T17:04:06.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-18T17:04:06.417 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-02-18T17:04:06.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T17:04:06.477 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-02-18T17:04:08.090 INFO:tasks.ceph.osd.0:Sent signal 15 2024-02-18T17:04:08.091 INFO:tasks.ceph.osd.1:Sent signal 15 2024-02-18T17:04:08.091 INFO:tasks.ceph.osd.2:Sent signal 15 2024-02-18T17:04:08.092 INFO:tasks.ceph.osd.4:Sent signal 15 2024-02-18T17:04:08.092 INFO:tasks.ceph.osd.5:Sent signal 15 2024-02-18T17:04:08.092 INFO:tasks.ceph.osd.6:Sent signal 15 2024-02-18T17:04:08.092 INFO:tasks.ceph.osd.7:Sent signal 15 2024-02-18T17:04:08.092 INFO:tasks.ceph.mds.a:Sent signal 15 2024-02-18T17:04:08.093 INFO:tasks.ceph.mds.c:Sent signal 15 2024-02-18T17:04:08.093 INFO:tasks.ceph.mds.b:Sent signal 15 2024-02-18T17:04:08.093 INFO:tasks.ceph.mds.d:Sent signal 15 2024-02-18T17:04:08.093 INFO:tasks.ceph.mon.a:Sent signal 15 2024-02-18T17:04:08.093 INFO:tasks.ceph.mon.b:Sent signal 15 2024-02-18T17:04:08.093 INFO:tasks.ceph.mon.c:Sent signal 15 2024-02-18T17:04:08.094 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-02-18T17:04:08.094 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-02-18T17:04:08.094 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T17:04:08.090+0000 7f2d640d1640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 38482) UID: 0 2024-02-18T17:04:08.095 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T17:04:08.090+0000 7f2d640d1640 -1 osd.4 42 *** Got signal Terminated *** 2024-02-18T17:04:08.095 INFO:tasks.ceph.osd.4.smithi143.stderr:2024-02-18T17:04:08.090+0000 7f2d640d1640 -1 osd.4 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.095 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T17:04:08.091+0000 7ff399284640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 38487) UID: 0 2024-02-18T17:04:08.095 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fed528fa640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 38485) UID: 0 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fed528fa640 -1 osd.6 42 *** Got signal Terminated *** 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.6.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fed528fa640 -1 osd.6 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T17:04:08.091+0000 7ff399284640 -1 osd.7 42 *** Got signal Terminated *** 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.7.smithi143.stderr:2024-02-18T17:04:08.091+0000 7ff399284640 -1 osd.7 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fe4fc5cc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 38481) UID: 0 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fe4fc5cc640 -1 osd.5 42 *** Got signal Terminated *** 2024-02-18T17:04:08.096 INFO:tasks.ceph.osd.5.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fe4fc5cc640 -1 osd.5 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.097 INFO:tasks.ceph.mds.b.smithi143.stderr:2024-02-18T17:04:08.091+0000 7f7bb0368640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 45112) UID: 0 2024-02-18T17:04:08.097 INFO:tasks.ceph.mds.b.smithi143.stderr:2024-02-18T17:04:08.091+0000 7f7bb0368640 -1 mds.b *** got signal Terminated *** 2024-02-18T17:04:08.097 INFO:tasks.ceph.mds.d.smithi143.stderr:2024-02-18T17:04:08.091+0000 7f3b2885d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 45113) UID: 0 2024-02-18T17:04:08.097 INFO:tasks.ceph.mds.d.smithi143.stderr:2024-02-18T17:04:08.091+0000 7f3b2885d640 -1 mds.d *** got signal Terminated *** 2024-02-18T17:04:08.098 INFO:tasks.ceph.mon.b.smithi143.stderr:2024-02-18T17:04:08.091+0000 7fb975580640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 37911) UID: 0 2024-02-18T17:04:08.098 INFO:tasks.ceph.mon.b.smithi143.stderr:2024-02-18T17:04:08.092+0000 7fb975580640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-02-18T17:04:08.098 INFO:tasks.ceph.mon.c.smithi143.stderr:2024-02-18T17:04:08.092+0000 7f57cd0f6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 37914) UID: 0 2024-02-18T17:04:08.098 INFO:tasks.ceph.mon.c.smithi143.stderr:2024-02-18T17:04:08.092+0000 7f57cd0f6640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-02-18T17:04:08.099 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T17:04:08.088+0000 7f9b36a5c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 38044) UID: 0 2024-02-18T17:04:08.099 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T17:04:08.088+0000 7f9b36a5c640 -1 osd.0 42 *** Got signal Terminated *** 2024-02-18T17:04:08.099 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T17:04:08.088+0000 7f348c2aa640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 38047) UID: 0 2024-02-18T17:04:08.099 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T17:04:08.088+0000 7f348c2aa640 -1 osd.1 42 *** Got signal Terminated *** 2024-02-18T17:04:08.099 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T17:04:08.093+0000 7f627de19640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 38049) UID: 0 2024-02-18T17:04:08.100 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T17:04:08.093+0000 7f30f177b640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 52180) UID: 0 2024-02-18T17:04:08.100 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T17:04:08.093+0000 7f627de19640 -1 osd.2 42 *** Got signal Terminated *** 2024-02-18T17:04:08.100 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-02-18T17:04:08.093+0000 7f8c06269640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 37780) UID: 0 2024-02-18T17:04:08.100 INFO:tasks.ceph.mon.a.smithi116.stderr:2024-02-18T17:04:08.093+0000 7f8c06269640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-18T17:04:08.101 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T17:04:08.093+0000 7fcaeff29640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 52179) UID: 0 2024-02-18T17:04:08.106 INFO:tasks.ceph.osd.0.smithi116.stderr:2024-02-18T17:04:08.102+0000 7f9b36a5c640 -1 osd.0 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.106 INFO:tasks.ceph.osd.1.smithi116.stderr:2024-02-18T17:04:08.102+0000 7f348c2aa640 -1 osd.1 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.106 INFO:tasks.ceph.osd.2.smithi116.stderr:2024-02-18T17:04:08.102+0000 7f627de19640 -1 osd.2 42 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T17:04:08.116 INFO:tasks.ceph.mds.c.smithi116.stderr:2024-02-18T17:04:08.112+0000 7f30f177b640 -1 mds.c *** got signal Terminated *** 2024-02-18T17:04:08.116 INFO:tasks.ceph.mds.a.smithi116.stderr:2024-02-18T17:04:08.112+0000 7fcaeff29640 -1 mds.a *** got signal Terminated *** 2024-02-18T17:04:08.297 INFO:tasks.ceph.mgr.x.smithi143.stderr:daemon-helper: command crashed with signal 15 2024-02-18T17:05:20.836 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:05:20.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:05:21.011 INFO:teuthology.orchestra.run.smithi143.stderr:ceph-osd: no process found 2024-02-18T17:08:36.538 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:08:36.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:09:06.632 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:09:06.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:09:36.703 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:09:36.707 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:10:06.824 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:10:06.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:10:36.971 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:10:36.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:11:07.127 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:11:07.130 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:11:37.198 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:11:37.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:12:07.261 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:12:07.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:12:37.399 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:12:37.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:13:07.546 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:13:07.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:13:37.704 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:13:37.708 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:14:07.958 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:14:07.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:14:38.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:14:38.031 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:15:08.150 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:15:08.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:15:38.280 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:15:38.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:16:08.426 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:16:08.429 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:16:38.572 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:16:38.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:17:08.700 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:17:08.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:17:38.914 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:17:38.918 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:18:08.994 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:18:08.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:18:39.274 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:18:39.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:19:09.412 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:19:09.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:19:39.701 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:19:39.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:20:09.767 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:20:09.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:20:39.939 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:20:39.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:21:10.057 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:21:10.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:21:40.130 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:21:40.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:22:10.259 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:22:10.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:22:40.389 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:22:40.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:23:10.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:23:10.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:23:40.623 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:23:40.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:24:10.694 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:24:10.697 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:24:40.757 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:24:40.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:25:10.994 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:25:10.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:25:41.139 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:25:41.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:26:11.210 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:26:11.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:26:41.343 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:26:41.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:27:11.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:27:11.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:27:41.481 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:27:41.484 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:28:11.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:28:11.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:28:41.993 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:28:41.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:29:12.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:29:12.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:29:42.447 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:29:42.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:30:12.610 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:30:12.614 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:30:42.691 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:30:42.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:31:12.837 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:31:12.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:31:42.966 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:31:42.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:32:13.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:32:13.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:32:43.393 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:32:43.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:33:13.539 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:33:13.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:33:43.685 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:33:43.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:34:13.756 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:34:13.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:34:43.903 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:34:43.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:35:14.050 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:35:14.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:35:44.192 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:35:44.195 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:36:14.336 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:36:14.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:36:44.706 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:36:44.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:37:14.986 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:37:14.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:37:45.057 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:37:45.061 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:38:15.154 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:38:15.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:38:45.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:38:45.341 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:39:15.695 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:39:15.698 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:39:45.982 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:39:45.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:40:16.040 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:40:16.044 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:40:46.177 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:40:46.181 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:41:16.353 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:41:16.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:41:46.456 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:41:46.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:42:16.619 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:42:16.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:42:46.700 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:42:46.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:43:16.922 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:43:16.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:43:47.060 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:43:47.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:44:17.197 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:44:17.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:44:47.270 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:44:47.273 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:45:17.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:45:17.428 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:45:47.575 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:45:47.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:46:17.646 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:46:17.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:46:47.714 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:46:47.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:47:17.812 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:47:17.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:47:48.084 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:47:48.087 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:48:18.153 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:48:18.157 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:48:48.334 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:48:48.338 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:49:18.504 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:49:18.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:49:48.569 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:49:48.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:50:18.633 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:50:18.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:50:48.734 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:50:48.738 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:51:18.821 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:51:18.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:51:49.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:51:49.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:52:19.247 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:52:19.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:52:49.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:52:49.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:53:19.446 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:53:19.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:53:49.568 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:53:49.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:54:19.806 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:54:19.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:54:49.879 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:54:49.882 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:55:20.001 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:55:20.006 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:55:50.121 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:55:50.125 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:56:20.211 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:56:20.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:56:50.331 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:56:50.335 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:57:20.522 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:57:20.525 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:57:50.659 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:57:50.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:58:20.739 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:58:20.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:58:50.860 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:58:50.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:59:21.025 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:59:21.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:59:51.179 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T17:59:51.182 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:00:21.314 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:00:21.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:00:51.605 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:00:51.608 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:01:21.677 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:01:21.680 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:01:51.845 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:01:51.849 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:02:21.990 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:02:21.993 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:02:52.217 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:02:52.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:03:22.296 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:03:22.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:03:52.460 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:03:52.463 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:04:22.540 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:04:22.544 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:04:52.612 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:04:52.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:05:22.741 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:05:22.745 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:05:52.871 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:05:52.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:06:22.935 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:06:22.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:06:53.081 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:06:53.087 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:07:23.345 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:07:23.349 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:07:53.520 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:07:53.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:08:23.692 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:08:23.695 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:08:53.973 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:08:53.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:09:24.173 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:09:24.177 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:09:54.244 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:09:54.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:10:24.473 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:10:24.476 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:10:54.753 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:10:54.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:11:24.819 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:11:24.822 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:11:54.970 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:11:54.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:12:25.098 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:12:25.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:12:55.241 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:12:55.247 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:13:25.317 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:13:25.320 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:13:55.455 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:13:55.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:14:25.725 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:14:25.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:14:55.879 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:14:55.883 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:15:26.150 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:15:26.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:15:56.230 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:15:56.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:16:26.394 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:16:26.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:16:56.530 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:16:56.534 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:17:26.660 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:17:26.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:17:56.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:17:56.750 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:18:26.826 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:18:26.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:18:57.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:18:57.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:19:27.199 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:19:27.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:19:57.353 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:19:57.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:20:27.518 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:20:27.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:20:57.663 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:20:57.667 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:21:27.927 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:21:27.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:21:58.066 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:21:58.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:22:28.221 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:22:28.225 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:22:58.537 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:22:58.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:23:28.651 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:23:28.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:23:58.764 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:23:58.769 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:24:29.077 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:24:29.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:24:59.154 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:24:59.157 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:25:29.326 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:25:29.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:25:59.613 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:25:59.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:26:29.709 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:26:29.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:26:59.782 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:26:59.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:27:30.128 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:27:30.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:28:00.243 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:28:00.247 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:28:30.397 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:28:30.401 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:29:00.678 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:29:00.681 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:29:30.851 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:29:30.855 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:30:00.995 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:30:00.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:30:31.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:30:31.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:31:01.401 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:31:01.405 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:31:31.470 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:31:31.475 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:32:01.547 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:32:01.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:32:31.672 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:32:31.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:33:01.801 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:33:01.805 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:33:31.874 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:33:31.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:34:02.012 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:34:02.020 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:34:32.151 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:34:32.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:35:02.247 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:35:02.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:35:32.369 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:35:32.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:36:02.515 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:36:02.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:36:32.662 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:36:32.665 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:37:02.725 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:37:02.729 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:37:32.860 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:37:32.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:38:03.010 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:38:03.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:38:33.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:38:33.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:39:03.211 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:39:03.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:39:33.359 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:39:33.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:40:03.497 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:40:03.500 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:40:33.611 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:40:33.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:41:03.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:41:03.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:41:33.820 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:41:33.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:42:03.983 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:42:03.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:42:34.228 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:42:34.231 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:43:04.495 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:43:04.498 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:43:34.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:43:34.561 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:44:04.640 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:44:04.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:44:35.001 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:44:35.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:45:05.139 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:45:05.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:45:35.225 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:45:35.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:46:05.363 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:46:05.367 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:46:35.508 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:46:35.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:47:05.662 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:47:05.666 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:47:35.822 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:47:35.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:48:06.084 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:48:06.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:48:36.207 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:48:36.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:49:06.282 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:49:06.287 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:49:36.436 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:49:36.439 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:50:06.638 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:50:06.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:50:36.832 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:50:36.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:51:06.904 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:51:06.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:51:36.976 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:51:36.979 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:52:07.046 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:52:07.050 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:52:37.208 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:52:37.211 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:53:07.271 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:53:07.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:53:37.409 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:53:37.413 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:54:07.564 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:54:07.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:54:37.634 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:54:37.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:55:07.772 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:55:07.776 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:55:37.922 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:55:37.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:56:08.042 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:56:08.045 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:56:38.261 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:56:38.265 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:57:08.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:57:08.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:57:38.503 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:57:38.506 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:58:08.574 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:58:08.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:58:38.644 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:58:38.648 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:59:08.717 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:59:08.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:59:38.847 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T18:59:38.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:00:09.016 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:00:09.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:00:39.151 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:00:39.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:01:09.477 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:01:09.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:01:39.654 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:01:39.657 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:02:09.789 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:02:09.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:02:40.059 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:02:40.063 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:03:10.252 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:03:10.255 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:03:40.375 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:03:40.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:04:10.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:04:10.500 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:04:40.643 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:04:40.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:05:10.764 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:05:10.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:05:40.907 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:05:40.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:06:11.061 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:06:11.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:06:41.195 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:06:41.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:07:11.320 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:07:11.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:07:41.499 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:07:41.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:08:11.610 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:08:11.613 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:08:41.797 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:08:41.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:09:11.961 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:09:11.964 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:09:42.283 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:09:42.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:10:12.437 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:10:12.441 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:10:42.600 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:10:42.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:11:12.769 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:11:12.772 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:11:42.910 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:11:42.914 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:12:12.998 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:12:13.001 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:12:43.214 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:12:43.217 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:13:13.342 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:13:13.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:13:43.412 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:13:43.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:14:13.517 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:14:13.521 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:14:43.589 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:14:43.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:15:13.702 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:15:13.706 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:15:43.913 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:15:43.917 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:16:14.141 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:16:14.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:16:44.208 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:16:44.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:17:14.322 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:17:14.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:17:44.587 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:17:44.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:18:14.941 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:18:14.944 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:18:45.013 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:18:45.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:19:15.209 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:19:15.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:19:45.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:19:45.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:20:15.413 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:20:15.416 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:20:45.550 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:20:45.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:21:15.618 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:21:15.622 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:21:45.757 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:21:45.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:22:15.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:22:15.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:22:46.024 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:22:46.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:23:16.277 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:23:16.280 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:23:46.384 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:23:46.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:24:16.532 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:24:16.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:24:46.671 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:24:46.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:25:16.894 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:25:16.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:25:47.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:25:47.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:26:17.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:26:17.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:26:47.538 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:26:47.542 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:27:17.659 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:27:17.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:27:47.816 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:27:47.819 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:28:17.890 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:28:17.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:28:48.036 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:28:48.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:29:18.191 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:29:18.194 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:29:48.262 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:29:48.266 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:30:18.387 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:30:18.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:30:48.542 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:30:48.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:31:18.614 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:31:18.617 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:31:48.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:31:48.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:32:18.900 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:32:18.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:32:48.973 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:32:48.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:33:19.038 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:33:19.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:33:49.299 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:33:49.303 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:34:19.439 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:34:19.442 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:34:49.569 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:34:49.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:35:19.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:35:19.726 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:35:49.872 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:35:49.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:36:20.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:36:20.010 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:36:50.161 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:36:50.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:37:20.315 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:37:20.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:37:50.445 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:37:50.448 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:38:20.566 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:38:20.570 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:38:50.695 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:38:50.698 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:39:21.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:39:21.071 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:39:51.196 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:39:51.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:40:21.342 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:40:21.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:40:51.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:40:51.499 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:41:21.642 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:41:21.645 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:41:51.771 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:41:51.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:42:21.918 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:42:21.921 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:42:52.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:42:52.070 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:43:22.343 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:43:22.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:43:52.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:43:52.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:44:22.543 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:44:22.546 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:44:52.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:44:52.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:45:22.843 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:45:22.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:45:53.009 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:45:53.013 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:46:23.135 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:46:23.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:46:53.272 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:46:53.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:47:23.444 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:47:23.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:47:53.597 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:47:53.600 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:48:23.742 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:48:23.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:48:53.897 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:48:53.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:49:24.070 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:49:24.074 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:49:54.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:49:54.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:50:24.327 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:50:24.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:50:54.465 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:50:54.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:51:24.618 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:51:24.622 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:51:54.764 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:51:54.768 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:52:24.942 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:52:24.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:52:55.081 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:52:55.084 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:53:25.227 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:53:25.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:53:55.381 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:53:55.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:54:25.537 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:54:25.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:54:55.713 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:54:55.716 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:55:25.870 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:55:25.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:55:55.995 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:55:55.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:56:26.129 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:56:26.132 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:56:56.275 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:56:56.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:57:26.440 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:57:26.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:57:56.575 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:57:56.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:58:26.696 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:58:26.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:58:56.833 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:58:56.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:59:26.978 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:59:26.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:59:57.142 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T19:59:57.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:00:27.291 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:00:27.294 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:00:57.424 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:00:57.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:01:27.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:01:27.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:01:57.683 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:01:57.687 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:02:27.812 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:02:27.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:02:57.960 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:02:57.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:03:28.092 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:03:28.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:03:58.246 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:03:58.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:04:28.392 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:04:28.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:04:58.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:04:58.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:05:28.700 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:05:28.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:05:58.845 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:05:58.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:06:28.975 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:06:28.979 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:06:59.127 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:06:59.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:07:29.259 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:07:29.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:07:59.405 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:07:59.408 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:08:29.536 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:08:29.539 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:08:59.691 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:08:59.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:09:29.822 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:09:29.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:09:59.943 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:09:59.946 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:10:30.065 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:10:30.757 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:11:00.902 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:11:00.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:11:31.032 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:11:31.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:12:01.143 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:12:01.146 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:12:31.282 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:12:31.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:13:01.418 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:13:01.421 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:13:31.563 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:13:31.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:14:01.697 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:14:01.701 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:14:31.838 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:14:31.841 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:15:01.975 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:15:01.978 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:15:32.129 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:15:32.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:16:02.283 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:16:02.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:16:32.412 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:16:32.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:17:02.559 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:17:02.562 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:17:32.686 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:17:32.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:18:02.832 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:18:02.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:18:32.953 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:18:32.957 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:19:03.074 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:19:03.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:19:33.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:19:33.229 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:20:03.356 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:20:03.359 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:20:33.536 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:20:33.539 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:21:03.666 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:21:03.670 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:21:33.812 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:21:33.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:22:03.884 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:22:03.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:22:34.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:22:34.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:23:04.145 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:23:04.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:23:34.249 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:23:34.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:24:04.337 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:24:04.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:24:34.467 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:24:34.473 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:25:04.596 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:25:04.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:25:34.742 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:25:34.745 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:26:04.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:26:04.893 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:26:35.053 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:26:35.062 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:27:05.308 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:27:05.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:27:35.471 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:27:35.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:28:05.600 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:28:05.604 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:28:35.665 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:28:35.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:29:05.807 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:29:05.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:29:35.939 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:29:35.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:30:06.085 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:30:06.089 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:30:36.155 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:30:36.159 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:31:06.293 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:31:06.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:31:36.430 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:31:36.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:32:06.600 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:32:06.604 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:32:36.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:32:36.741 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:33:06.875 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:33:06.878 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:33:37.026 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:33:37.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:34:07.156 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:34:07.160 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:34:37.302 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:34:37.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:35:07.456 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:35:07.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:35:37.586 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:35:37.589 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:36:07.716 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:36:07.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:36:37.837 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:36:37.841 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:37:08.009 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:37:08.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:37:38.138 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:37:38.141 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:38:08.301 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:38:08.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:38:38.439 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:38:38.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:39:08.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:39:08.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:39:38.759 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:39:38.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:40:08.904 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:40:08.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:40:39.051 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:40:39.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:41:09.205 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:41:09.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:41:39.351 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:41:39.354 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:42:09.520 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:42:09.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:42:39.658 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:42:39.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:43:09.795 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:43:09.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:43:39.865 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:43:39.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:44:10.011 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:44:10.015 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:44:40.183 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:44:40.186 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:45:10.377 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:45:10.380 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:45:40.526 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:45:40.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:46:10.662 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:46:10.666 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:46:40.790 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:46:40.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:47:10.953 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:47:10.957 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:47:41.107 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:47:41.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:48:11.269 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:48:11.273 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:48:41.410 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:48:41.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:49:11.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:49:11.562 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:49:41.706 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:49:41.710 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:50:11.844 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:50:11.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:50:41.988 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:50:41.992 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:51:12.136 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:51:12.139 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:51:42.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:51:42.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:52:12.411 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:52:12.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:52:42.565 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:52:42.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:53:12.702 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:53:12.706 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:53:42.856 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:53:42.859 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:54:13.001 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:54:13.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:54:43.148 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:54:43.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:55:13.374 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:55:13.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:55:43.528 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:55:43.532 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:56:13.676 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:56:13.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:56:43.814 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:56:43.818 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:57:13.951 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:57:13.954 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:57:44.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:57:44.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:58:14.218 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:58:14.222 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:58:44.346 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:58:44.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:59:14.475 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:59:14.479 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:59:44.633 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T20:59:44.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:00:14.775 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:00:14.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:00:44.959 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:00:44.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:01:15.106 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:01:15.110 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:01:45.248 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:01:45.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:02:15.399 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:02:15.402 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:02:45.554 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:02:45.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:03:15.708 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:03:15.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:03:45.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:03:45.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:04:16.060 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:04:16.063 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:04:46.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:04:46.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:05:16.354 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:05:16.357 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:05:46.490 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:05:46.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:06:16.628 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:06:16.631 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:06:46.749 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:06:46.753 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:07:16.905 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:07:16.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:07:47.043 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:07:47.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:08:17.107 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:08:17.110 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:08:47.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:08:47.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:09:17.306 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:09:17.310 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:09:47.470 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:09:47.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:10:17.641 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:10:17.644 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:10:47.794 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:10:47.797 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:11:17.984 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:11:17.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:11:48.111 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:11:48.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:12:18.257 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:12:18.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:12:48.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:12:48.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:13:18.552 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:13:18.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:13:48.708 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:13:48.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:14:18.839 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:14:18.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:14:48.974 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:14:48.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:15:19.114 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:15:19.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:15:49.242 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:15:49.245 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:16:19.573 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:16:19.576 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:16:49.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:16:49.764 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:17:19.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:17:19.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:17:50.033 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:17:50.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:18:20.170 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:18:20.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:18:50.238 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:18:50.241 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:19:20.335 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:19:20.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:19:50.489 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:19:50.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:20:20.554 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:20:20.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:20:50.632 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:20:50.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:21:20.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:21:20.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:21:50.900 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:21:50.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:22:21.055 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:22:21.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:22:51.184 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:22:51.188 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:23:21.314 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:23:21.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:23:51.444 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:23:51.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:24:21.565 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:24:21.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:24:51.695 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:24:51.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:25:21.767 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:25:21.770 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:25:51.913 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:25:51.917 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:26:22.042 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:26:22.046 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:26:52.190 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:26:52.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:27:22.277 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:27:22.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:27:52.549 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:27:52.552 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:28:22.687 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:28:22.692 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:28:52.817 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:28:52.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:29:22.981 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:29:22.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:29:53.110 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:29:53.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:30:23.298 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:30:23.302 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:30:53.369 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:30:53.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:31:23.458 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:31:23.462 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:31:53.663 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:31:53.666 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:32:23.827 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:32:23.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:32:53.897 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:32:53.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:33:23.969 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:33:23.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:33:54.099 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:33:54.102 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:34:24.254 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:34:24.257 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:34:54.417 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:34:54.421 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:35:24.571 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:35:24.574 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:35:54.636 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:35:54.639 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:36:24.780 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:36:24.784 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:36:54.926 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:36:54.930 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:37:25.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:37:25.077 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:37:55.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:37:55.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:38:25.390 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:38:25.393 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:38:55.519 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:38:55.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:39:25.656 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:39:25.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:39:55.728 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:39:55.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:40:25.798 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:40:25.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:40:55.936 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:40:55.940 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:41:26.008 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:41:26.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:41:56.145 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:41:56.149 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:42:26.200 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:42:26.204 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:42:56.336 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:42:56.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:43:26.526 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:43:26.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:43:56.673 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:43:56.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:44:26.788 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:44:26.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:44:56.939 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:44:56.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:45:27.077 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:45:27.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:45:57.198 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:45:57.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:46:27.336 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:46:27.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:46:57.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:46:57.485 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:47:27.635 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:47:27.638 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:47:57.772 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:47:57.775 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:48:27.917 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:48:27.921 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:48:58.046 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:48:58.049 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:49:28.175 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:49:28.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:49:58.312 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:49:58.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:50:28.465 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:50:28.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:50:58.611 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:50:58.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:51:28.765 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:51:28.768 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:51:58.877 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:51:58.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:52:29.006 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:52:29.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:52:59.152 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:52:59.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:53:29.289 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:53:29.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:53:59.451 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:53:59.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:54:29.586 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:54:29.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:54:59.717 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:54:59.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:55:29.871 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:55:29.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:56:00.133 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:56:00.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:56:30.279 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:56:30.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:57:00.432 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:57:00.435 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:57:30.553 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:57:30.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:58:00.690 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:58:00.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:58:30.840 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:58:30.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:59:01.000 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:59:01.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:59:31.136 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T21:59:31.139 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:00:01.273 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:00:01.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:00:31.426 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:00:31.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:01:01.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:01:01.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:01:31.685 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:01:31.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:02:01.805 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:02:01.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:02:31.943 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:02:31.947 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:03:02.095 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:03:02.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:03:32.250 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:03:32.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:04:02.370 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:04:02.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:04:32.513 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:04:32.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:05:02.651 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:05:02.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:05:32.790 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:05:32.793 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:06:02.927 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:06:02.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:06:33.057 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:06:33.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:07:03.218 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:07:03.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:07:33.363 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:07:33.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:08:03.509 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:08:03.512 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:08:33.652 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:08:33.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:09:03.823 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:09:03.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:09:33.961 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:09:33.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:10:04.101 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:10:04.105 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:10:34.280 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:10:34.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:11:04.408 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:11:04.411 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:11:34.571 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:11:34.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:12:04.754 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:12:04.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:12:34.900 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:12:34.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:13:05.055 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:13:05.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:13:35.187 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:13:35.191 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:14:05.342 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:14:05.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:14:35.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:14:35.499 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:15:05.676 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:15:05.680 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:15:35.820 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:15:35.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:16:05.932 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:16:05.935 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:16:36.094 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:16:36.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:17:06.233 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:17:06.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:17:36.386 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:17:36.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:18:06.532 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:18:06.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:18:36.686 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:18:36.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:19:06.840 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:19:06.844 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:19:36.986 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:19:36.989 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:20:07.131 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:20:07.135 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:20:37.268 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:20:37.272 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:21:07.555 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:21:07.559 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:21:37.727 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:21:37.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:22:07.889 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:22:07.892 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:22:38.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:22:38.021 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:23:08.155 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:23:08.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:23:38.292 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:23:38.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:24:08.447 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:24:08.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:24:38.592 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:24:38.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:25:08.737 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:25:08.741 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:25:38.861 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:25:38.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:26:09.030 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:26:09.033 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:26:39.173 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:26:39.177 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:27:09.328 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:27:09.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:27:39.449 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:27:39.452 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:28:09.612 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:28:09.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:28:39.748 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:28:39.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:29:09.886 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:29:09.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:29:40.048 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:29:40.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:30:10.168 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:30:10.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:30:40.322 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:30:40.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:31:10.692 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:31:10.696 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:31:40.848 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:31:40.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:32:10.916 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:32:10.919 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:32:41.038 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:32:41.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:33:11.167 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:33:11.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:33:41.312 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:33:41.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:34:11.467 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:34:11.471 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:34:41.612 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:34:41.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:35:11.750 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:35:11.754 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:35:41.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:35:41.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:36:12.016 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:36:12.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:36:42.162 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:36:42.165 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:37:12.301 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:37:12.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:37:42.445 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:37:42.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:38:12.598 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:38:12.601 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:38:42.744 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:38:42.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:39:13.060 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:39:13.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:39:43.215 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:39:43.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:40:13.280 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:40:13.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:40:43.342 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:40:43.345 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:41:13.405 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:41:13.408 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:41:43.509 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:41:43.513 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:42:13.665 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:42:13.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:42:43.794 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:42:43.797 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:43:13.857 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:43:13.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:43:43.987 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:43:43.991 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:44:14.129 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:44:14.132 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:44:44.263 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:44:44.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:45:14.402 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:45:14.406 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:45:44.517 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:45:44.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:46:14.705 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:46:14.708 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:46:44.768 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:46:44.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:47:14.899 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:47:14.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:47:44.971 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:47:44.994 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:48:15.063 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:48:15.067 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:48:45.203 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:48:45.207 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:49:15.288 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:49:15.291 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:49:45.439 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:49:45.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:50:15.590 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:50:15.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:50:45.736 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:50:45.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:51:16.000 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:51:16.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:51:46.071 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:51:46.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:52:16.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:52:16.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:52:46.373 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:52:46.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:53:16.486 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:53:16.490 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:53:46.616 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:53:46.620 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:54:16.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:54:16.742 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:54:46.812 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:54:46.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:55:16.874 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:55:16.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:55:46.945 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:55:46.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:56:17.092 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:56:17.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:56:47.222 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:56:47.226 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:57:17.353 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:57:17.648 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:57:47.773 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:57:47.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:58:17.895 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:58:17.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:58:48.026 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:58:48.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:59:18.164 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:59:18.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:59:48.310 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T22:59:48.314 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:00:18.449 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:00:18.452 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:00:48.521 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:00:48.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:01:18.607 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:01:18.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:01:48.673 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:01:48.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:02:18.777 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:02:18.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:02:48.907 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:02:48.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:03:19.021 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:03:19.025 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:03:49.151 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:03:49.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:04:19.289 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:04:19.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:04:49.369 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:04:49.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:05:19.441 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:05:19.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:05:49.579 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:05:49.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:06:19.734 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:06:20.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:06:50.138 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:06:50.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:07:20.211 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:07:20.214 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:07:50.366 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:07:50.369 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:08:20.521 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:08:20.525 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:08:50.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:08:50.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:09:20.756 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:09:20.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:09:50.894 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:09:50.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:10:21.039 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:10:21.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:10:51.202 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:10:51.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:11:21.372 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:11:21.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:11:51.521 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:11:51.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:12:21.672 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:12:21.698 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:12:51.843 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:12:51.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:13:22.103 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:13:22.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:13:52.185 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:13:52.188 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:14:22.328 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:14:22.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:14:52.392 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:14:52.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:15:22.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:15:22.468 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:15:52.602 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:15:52.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:16:22.749 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:16:22.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:16:52.862 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:16:52.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:17:23.017 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:17:23.020 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:17:53.164 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:17:53.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:18:23.277 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:18:23.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:18:53.407 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:18:53.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:19:23.545 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:19:23.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:19:53.683 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:19:53.686 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:20:23.754 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:20:23.758 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:20:53.894 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:20:53.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:21:24.014 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:21:24.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:21:54.152 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:21:54.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:22:24.332 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:22:24.336 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:22:54.469 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:22:54.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:23:24.632 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:23:24.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:23:54.745 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:23:54.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:24:24.817 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:24:24.821 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:24:54.879 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:24:54.883 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:25:24.951 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:25:24.954 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:25:55.022 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:25:55.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:26:25.152 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:26:25.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:26:55.214 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:26:55.217 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:27:25.289 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:27:25.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:27:55.349 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:27:55.353 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:28:25.496 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:28:25.499 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:28:55.643 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:28:55.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:29:25.779 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:29:25.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:29:55.918 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:29:55.921 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:30:26.056 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:30:26.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:30:56.278 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:30:56.281 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:31:26.390 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:31:26.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:31:56.520 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:31:56.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:32:26.594 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:32:26.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:32:56.656 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:32:56.660 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:33:26.726 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:33:26.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:33:56.814 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:33:56.824 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:34:26.885 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:34:26.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:34:57.023 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:34:58.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:35:28.667 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:35:28.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:35:58.786 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:35:58.790 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:36:28.932 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:36:28.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:36:59.003 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:36:59.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:37:29.141 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:37:29.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:37:59.288 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:37:59.311 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:38:29.450 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:38:29.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-02-18T23:38:37.479 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-02-18T23:38:37.480 DEBUG:teuthology.task.console_log:Killing console logger for smithi116 2024-02-18T23:38:37.481 DEBUG:teuthology.task.console_log:Killing console logger for smithi143 2024-02-18T23:38:37.481 DEBUG:teuthology.exit:Finished running handlers